What Search Engine Spiders See

Posted June 15, 2006 at 9:57 AM by Ben Nadel

Tags: Search Engine Optimization

I was just posting a Search Engine Optimization question on the House of Fusion CF-Talk list when Nick Tong posted a link to "What Search Engine Spiders See". This page blows my mind. It allows you to enter in a url and get a report back regarding the page content based on how the search engine web spiders/crawlers see your content. It is fantastic for seeing exactly what your page has and perhaps more importantly, what your page is lacking in terms of headers, alt tags, and general content.

Nick posted this saying that he originally got it off of AndyJarret.co.uk.

Link: http://www.dlperry.com/what_search_engine_spiders_see.html



Reader Comments

Wong Seoul
Jul 21, 2007 at 10:35 PM // reply »
1 Comments
1 Points

Great tool. I always use crawler simulation tools from seochat. This one looks much better and detailed.

Thanks for sharing


Jan
Sep 6, 2007 at 3:06 AM // reply »
6 Comments
1 Points

There is one thing which developers forgot about; Comments. I just checked one of my sites which has few comments (HTML tags of course) and it didn't remove the content within comments. Google doesn't consider comments for indexing.

In this sense, comments may contain links, words, whatever... So the tool may give you invalid output in some cases.


Florida Search Engine Optimization
May 6, 2008 at 10:48 AM // reply »
9 Comments
1 Points

Nice tool ...

Check this out too ...

http://www.ranks.nl/tools/spider.html


Ben Nadel
May 6, 2008 at 1:01 PM // reply »
10,640 Comments
69,065 Points

@Edward,

Looks interesting. My pages are too big to use with the free version :)


ana gomez
May 30, 2008 at 9:49 AM // reply »
2 Comments
1 Points

I could not find that tool on that site. Do i have to regiter to use it?


Post A Comment

Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.

Please review the following issues:

Author Name:
Author Email:
Author Website:
Comment:
Supported HTML tags for formatting: <strong>bold</strong>   <em>italic</em>   <code>code</code>







  • Help Wanted - Find Your Next ColdFusion Job
InVision App - Prototyping Made Beautiful With Prototyping Tools Ben Nadel's Company - Epicenter Consulting Recent Blog Comments
Bartezz
Feb 10, 2012 at 7:21 PM
jQuery AJAX Strips Script Tags And Inserts Them After Parent-Most Elements
Update! Instead of $(eval(options.insertAfter)).after(data['insertData']); I now use: var ajaxNode = document.createElement('span'); var parent = $(eval(options.insertAfter))[0].parentNode; ... read »
Bartezz
Feb 10, 2012 at 6:18 PM
jQuery AJAX Strips Script Tags And Inserts Them After Parent-Most Elements
encountered this same, what I consider, jQuery bug last week. I'm building a site in which I load some content via AJAX. This content contains Linkedin share button placeholders which Linkedin API ne ... read »
Drew Wells
Feb 10, 2012 at 11:30 AM
Cross-Origin Resource Sharing (CORS) AJAX Requests Between jQuery And Node.js
After you understand the concepts here, this is an awesome cheatsheet for enabling CORS in just about anything http://enable-cors.org/ ... read »
JM
Feb 10, 2012 at 9:10 AM
My Safari Browser SQLite Database Hello World Example
@Amy, Here is a very good tutorial on how to use JOIN: http://www.sqltutorial.org/sqljoin-innerjoin.aspx ... read »
Stefan Richter
Feb 10, 2012 at 4:42 AM
Building A Twitter-Inspired RESTful API Architecture In ColdFusion
This is great, very useful Ben. I spotted a small typo in the api.cgm listing: <cfthrow type="Unauthroized" /> Cheers Stefan ... read »
Hugh
Feb 9, 2012 at 10:35 PM
CFDirectory Filtering Uses Pipe Character For Multiple Filters (Thanks Steve Withington)
I was wondering if there would be a filter you could apply so that you got everything but what you included in the filter. As in show me all docs that are not a .pdf. ... read »
Stephen
Feb 9, 2012 at 10:29 PM
Learning ColdFusion 9: Application-Specific Data Sources
@Ben, No offence, but if people were really wanting advanced features they would be using a platform like ASP.NET MVC. CFML is so structurally compromised as a tag-based scripting language that ... read »
Nathan
Feb 9, 2012 at 10:03 PM
Subversion - Cleanup Failed To Process The Following Paths
@Leviaguirre, do you still have problems with this? ... read »