What Search Engine Spiders See

By Ben Nadel

Published 2006-06-15 in Search Engine Optimization — Comments (5)

I was just posting a Search Engine Optimization question on the House of Fusion CF-Talk list when Nick Tong posted a link to "What Search Engine Spiders See". This page blows my mind. It allows you to enter in a url and get a report back regarding the page content based on how the search engine web spiders/crawlers see your content. It is fantastic for seeing exactly what your page has and perhaps more importantly, what your page is lacking in terms of headers, alt tags, and general content.

Nick posted this saying that he originally got it off of AndyJarret.co.uk.

Link: http://www.dlperry.com/what_search_engine_spiders_see.html

Short link: https://bennadel.com/86

Reader Comments

Wong Seoul Jul 21, 2007 at 10:35 PM

1 Comments

Great tool. I always use crawler simulation tools from seochat. This one looks much better and detailed.

Thanks for sharing

Jan Sep 6, 2007 at 3:06 AM

6 Comments

There is one thing which developers forgot about; Comments. I just checked one of my sites which has few comments (HTML tags of course) and it didn't remove the content within comments. Google doesn't consider comments for indexing.

In this sense, comments may contain links, words, whatever... So the tool may give you invalid output in some cases.