Skin Spider : Results You Can Finally See
Posted October 3, 2006 at 3:05 PM by Ben Nadel
To learn all about Skin Spider, click here.
To view the most updated application, click here.
To view the current code base, click here.
Up until now, most of the Skin Spider updates have been to core files and spidering techniques. Now, I finally started working on pages the display content. Currently it's only the Home page and the preliminary Search page, but you can get an idea of how it is all going to come together.
Now, I like to have fun, but this is serious stuff. It might just be a general purpose ColdFusion application demo, but the content of the site is intended for mature audiences. To help ensure that people do not stumble on this accidentally, I have created an age verification page (age_verification.cfm) where users must confirm that they are at least 18 years of age or older. Now, this is JUST for the demo. I wouldn't leave this page in on a personal computer - that's just strange.
I am including this age verification page from the Application.cfm page to ensure that anything that goes through pre-page processing will have the chance to check age verification. However, currently, the system doesn't use any session management or client cookies so how do I tell if a user has already confirmed their age? For now, I am operating on the assumption that if you are in the site, you have confirmed and if you are COMING TO the site, you have not yet confirmed.
In ColdFusion, the CGI.http_referer value is the URL that the user was just on before they came to the current page. Since I want people coming to the application to verify their age, I will check to see if the http_referer value is outside the current application. If it is, then I include the age verification page and abort the rest of the page processing (so that nothing else but the age verification page shows). If they are coming from a page that IS in the current application then I assume they have already verified and I skip the age verification include and allow the page to continue processing as normal.
Since all of the pages are in the top level directory, I assume a person is in the application if the current page's parent directory is the same as the parent directory of the Application.cfm ColdFusion template. This of course will get more complicated if I have sub-directory pages, for now, though, this should work out fine. That sounds complicated but basically I am just making sure that the current page and the referer page are in the same directory and if not, they have to verify their age.
DatabaseService.cfc ColdFusion Component
I have finally updated my DatabaseService.cfc to allow a record to be updated. Up until now, you could only delete a record or get a table. Now, you can update a record by key using the UpdateRecordByKey() method. You must pass in the table name, the primary key, and the data structure. This is preliminary, but the Data structure being passed in is just an arbitrary struct. Once in the method, the system loops over the actual columns of the table and checks to see if a given column name lines up with a key in the data struct. If it does, it sets the value. If not, then it just moves on to check the next query column. The idea is that the record is only updated with the values that are sent. And, since we are looping over the query columns (not the data struct keys), we are helping to ensure that we don't try to alter the query structure.
I haven't tested this though, but I think the theory is sound. Actually, this makes me think about the GetNewRecord() method. This method returns an unpopulated struct of column_name keys. Now that I think about it, this is not really necessary. I am kind of liking the way that UpdateRecordByKey() only uses values passed in. I think I will probably move over to that methodology for the adding of records as well... we will see.
I have updated the spider_gallery.cfm page. In particular, I have updated the regular expression used for matching links to videos. Originally, it would match any URL that had something like .mov or .mpeg in it. This was causing parts of URLs to get matched. For example, http://www.movies-r-us.com was getting matches as "http://www.mov" since it was a non-greedy search looking to match on .mov. I have updated the regular expression to make sure that the movie file extension was at the end of the link (as followed by either a quote, space, or bracket). This seems to be working much better.
I am actually very impressed with how fast galleries and videos are spidered. Not that these videos are huge, but still, impressive.
Home Page / System Overview
I have gotten the preliminary home page up and running. This page merely lists out the most recently spidered videos and gives some hard drive statistics. To get the hard drive statistics, I am doing a CFDirectory call on the video directory. Then, to get the hard drive space, I simply call ArraySum() on the size query column. This is just a cool little example of how in ColdFusion you can call Array functions on ColdFusion query columns.
I have a preliminary search page up and running. Right now, all I have really concentrated on is the search results pagination. This is not going to be a good use-case for pagination due to the nature of the XML database. Pagination is really leveraged well when you don't have to get all the records back from the database. But, since our database is in-memory, the best benefits of pagination cannot be realized. Instead, my pagination works by calculating offsets and using StartRow and EndRow CFLoop attributes. Not the most elegant solution, but definitely worth a look.
In my pagination, I set the page size (number of results per page) and the bucket size (number of total pages to show in the pagination navigation). The way I have it set up now, the current page is always in the middle of the bucket, if possible.
Even though my search is very preliminary right now, I can already see how search criteria will become an issue. I have one possible piece of criteria, the gallery id, that can be used. This has to be put in all pagination links. This is ok for one or two criteria, but this is going to be pain in the butt as I add more criteria. This is something that will definitely have to be refactored in Phase II (as well as an encapsulation of the pagination).
You will notice that on the home page as well as on the search page, I try to use a video thumbnail. If that is not available I try to use a gallery thumbnail. If that is not available, I use the "no preview" thumbnail. The bottom line is that all videos will have some sort of thumbnail. This will keep a very consistent look and feel to the site.
What Other People Are Searching For
[ local search ]
coldfusion application demo
[ local search ] iterative approach to coldfusion application development
[ local search ] coldfusion application development
[ local search ] mx7 sample applications
[ local search ] cfmx sample applications
Just in case anyone else could not get past your age verification and it wasn't because I'm under 18 and kept clicking that button ;) - my symantec client firewall was by default blocking the http_referer from being passed from the browser.
Why do you have to make my life so complicated :) No problem. I will update the way it works. The homepage is technically "home.cfm". I will update it so that the index.cfm page pulls in the age verification page on the next code update.
This will force the age verification if no page (ex. search.cfm) is defined (since it defaults to index.cfm). However, this is easily gotten around by not going to the homepage... but oh well... it's just a demo.
Thanks for the heads up.