I am getting errors when I try to grab google results with cfhttp. But, when I go to page with my browser, it works just fine. What am I doing wrong?
You are not doing anything wrong. Google wants to be used by regular web users. CFHttp does not announce itself as a regular user. When you do a CFHttp page grab, it passes along, as its User Agent a non-standard value. I am not sure offhand what it is, but I think it sends "ColdFusion" as its user agent. Doing a regular CFHttp will return this error:
Your client does not have permission to get URL /search?hl=en&lr=&q=upsidedown+dogs&btnG=Search from this server.
This is there for a reason: you might be violating the Google terms of service (I have not read them, nor do I condone working around this). If you want to avoid this, you can fake Google into thinking you ARE a web browser by sending a standard user agent in your CFHttp:
<!--- Grab the google search results. ---> <cfhttp url="http://www.google.com/search?hl=en&lr=&q=upsidedown+dogs&btnG=Search" useragent="Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 2.0.50727; FDM)" result="objGoogleGrab" method="GET" resolveurl="true" /> <!--- Output the search results. ---> <cfoutput> #objGoogleGrab.FileContent# </cfoutput>
Notice that I am sending the FireFox / Mozilla user agent. This should work just fine. But again, I am not aware of the legality of such an action - proceed with caution.
Want to use code from this post? Check out the license.