Passing Referer AS ColdFusion CFHttp CGI Value vs HEADER Value?
Posted August 9, 2007 at 9:23 AM
The other day, someone posted a SPAM comment on my site that directed users to a site serving up free Playboy magazine centerfold downloads. I am so tired of people posting spam to my site that I figured I would actually try and turn the tables on this particular spammer. Sites like this operate on the fact that you have to actually view their pages and therefore, view their ads (which is how, I assume, they make their money). However, grabbing the merchandise without loading any ads defeats their entire reason for spamming in the first place.
And so it is that I set out to script the download of every Playboy magazine centerfold from 1954 to 2007 without ever loading a single page on the spammers site (naturally, I had to load a few pages to see how the site worked). Here's what I came up with:
Launch code in new window » Download code as text file »
- <!---
- Photo pages in form of:
- http://www.oxpe.net/playboy/playboy195401.html
-
- Photos in form of:
- http://www.oxpe.net/playboy/photos/195401.jpg
-
- Available Years:
- 1954 - 2007
- --->
-
- <!---
- Set a high request timeout - there are a LOT of
- images to be downloading here.
- --->
- <cfsetting requesttimeout="500" />
-
- <!--- Loop over available years. --->
- <cfloop
- index="intYear"
- from="1954"
- to="2007"
- step="1">
-
- <!--- Loop over available months. --->
- <cfloop
- index="intMonth"
- from="1"
- to="12"
- step="1">
-
- <!---
- Get the short hand for the file name. All the
- months have to be double-digits.
- --->
- <cfset strName = (
- intYear &
- NumberFormat( intMonth, "00" )
- ) />
-
- <!---
- Set up base URL that will be used by both the
- CFHttp target as well as the referer.
- --->
- <cfset strBaseURL = "http://www.oxpe.net/playboy/" />
-
-
- <!--- Echo back the photo we are trying to get. --->
- <p>
- <cfoutput>
- #strName#.jpg
- </cfoutput>
- </p>
-
-
- <!---
- Perform an HTTP GET to grab the target image
- as binary. CAUTION: Once we go beyond the
- vailable year/months (ex. 2007/12), this will
- come back with 200 status, but NOT be a valid
- image binary.
- --->
- <cfhttp
- method="get"
- url="#strBaseURL#photos/#strName#.jpg"
- useragent="#CGI.http_user_agent#"
- getasbinary="yes"
- result="objGET">
-
- <!---
- Set CGI referrer to be the page that it was
- called from. We want to fake the target
- server into thinking we just came from an
- internally hosted page.
- --->
- <cfhttpparam
- type="CGI"
- name="referer"
- value="#strBaseURL#playboy#strName#.html"
- />
-
- </cfhttp>
-
-
- <!--- Check status. --->
- <cfif FindNoCase( "200", objGET.StatusCode )>
-
- <!--- Save file. --->
- <cffile
- action="write"
- file="#ExpandPath( './#strName#.jpg' )#"
- output="#objGET.FileContent#"
- />
-
- </cfif>
-
-
- <p>
- <cfoutput>
- » <em>#objGET.StatusCode#</em>
- </cfoutput>
- </p>
-
- <cfflush />
-
- </cfloop>
-
- </cfloop>
Unfortunately, this did not work at all. Running the code above, the server kept returning 403 Forbidden Access errors:
195401.jpg
» 403 Forbidden
195402.jpg
» 403 Forbidden
195403.jpg
» 403 Forbidden
Clearly, the server had something in place to prevent hotlinking. But, I was sending the Referer, which should have taken care of this.
After a good deal of time trying to tweak the values, I finally turned to one of the most badass tools out there - FireBug. I actually went to the target page and viewed the HTTP Request headers that were being sent across for the graphic request:
| | | | ||
| |  | | ||
| | | |
Nothing was popping out at me. But then, I realized I was looking at the HEADER values. Obviously. But, wasn't I sending the Referer value as a CGI value? I know that ColdFusion's CFHttpParam tag has a HEADER type, so I tried to change the type from Referer to HEADER:
Launch code in new window » Download code as text file »
- <!---
- Perform an HTTP GET to grab the target image
- as binary. CAUTION: Once we go beyond the
- vailable year/months (ex. 2007/12), this will
- come back with 200 status, but NOT be a valid
- image binary.
- --->
- <cfhttp
- method="get"
- url="#strBaseURL#photos/#strName#.jpg"
- useragent="#CGI.http_user_agent#"
- getasbinary="yes"
- result="objGET">
-
- <!---
- Set referrer to be the page that it was called
- from. We want to fake the target server into
- thinking we just came from an internally hosted
- page. Use the HEADER value rather than the CGI
- value.
- --->
- <cfhttpparam
- type="HEADER"
- name="referer"
- value="#strBaseURL#playboy#strName#.html"
- />
-
- </cfhttp>
This time, things went off without a hitch:
95401.jpg
» 200 OK
195402.jpg
» 200 OK
195403.jpg
» 200 OK
Works great - downloads all the Playboy centerfolds since the beginning of time, but this got me thinking: if both the CGI:Referer and the HEADER:Referer end up in the CGI scope (at least in ColdFusion), what's the difference between sending these two values. Why did one work and one not work?
The answer turns out to be Encoding. By default, the ColdFusion CFHttpParam tag Encodes all FormField and CGI value types using a URL-encoding. HEADER values, on the other hand, are not encoded in any automatic way. Therefore, if you tried to send this value:
http://www.oxpe.net/playboy/playboy.html
... as a CFHttpParam CGI value, it would show up in the CGI object as:
http%3A%2F%2Fwww%2Eoxpe%2Enet%2Fplayboy%2Fplayboy%2Ehtml
This has been encoded for URL usage. If, however you turned off encoding (Encoded = "false"), ** OR ** sent it as a CFHttpParam HEADER value, it would show up in the CGI object as:
http://www.oxpe.net/playboy/playboy.html
This is good stuff to know. I should really do a much more in-depth exploration of all the different CFHttpParam types to see how they can really be leveraged properly. I am still not 100% clear on the difference between all the HEADER and CGI values; it looks like HEADER values might be a more natural way to mimic this sort of clint-server interaction. I will do some further testing.
On a related note, it's really funny to see the contrast in what Playboy presented in 1954 compared to what they put in the magazine in today. 1954 was very safe. Pubic hair didn't really make much of any show until the early 1970s... and now, in the 2000s, pubic hair is gone again (but this time, of course, for very different reasons).
| | | | ||
| |  | | ||
| | | |
Download Code Snippet ZIP File
Comments (19) | Post Comment | Ask Ben | Permalink | Other Searches | Print Page
Javascript's Implicit Boolean Conversions
Selecting Contrasting Text Color Based On Background Color
Reader Comments
Ben Nadel, ladies and gentlemen... always ready to tackle the topics that others want to know but are afraid to talk about.... like how to safely strip porn from spammer websites in the name of learning!
I love it! Now to download the code so I can investigate it further... for learning purposes, of course! ;)
Posted by Max on Aug 9, 2007 at 10:01 AM
@Max,
It's all about the learning :) I learned something very valuable here about using ColdFusion's CFHttp and CFHttpParam tags. Totally justified :)
Posted by Ben Nadel on Aug 9, 2007 at 10:11 AM
Yep... and I've always felt it learning was easier when passionate about the subject matter.
That subject being CF, y'know.... ummmm yeah, ColdFusion.
Posted by Max on Aug 9, 2007 at 10:17 AM
ColdFusion is dead sexy.
Posted by Ben Nadel on Aug 9, 2007 at 10:32 AM
He he, good job! I love the above banner too :-)
Posted by Boyan on Aug 9, 2007 at 4:18 PM
Ben for president!!!!!!!!!!!!!
Posted by Donnie Bachan on Aug 9, 2007 at 5:39 PM
I'm just doing my part in the war against spammers :)
Posted by Ben Nadel on Aug 9, 2007 at 5:42 PM
Just to let you know, I think this is great Ben! A little cf, some classic porn, eating a spammers bandwidth, I love.
I have 4 servers running it right now, just to hurt them! :-)
Posted by cfPorn Fan on Aug 9, 2007 at 10:07 PM
Ha ha ha ha. That reminds me of the movie Hackers where they had people all over the world attacking this one computer system. That was a sweet movie .... back when Angelina Jolie was actually acttractive to me :(
Posted by Ben Nadel on Aug 10, 2007 at 8:16 AM
Ben Nadel, you are a wicked, wicked man. And that is why we value you so highly in the CF community. :-)
Posted by Trent on Aug 10, 2007 at 8:09 PM
@Trent,
I'm just one dude trying to make a difference.
Posted by Ben Nadel on Aug 10, 2007 at 8:55 PM
Header values are the meta data that goes into the HTTP request (and return with the response). The receiving webserver turns header values into CGI variables, and CGI variables are available during request processing. I think of it like: HEADER > Webserver > CGI > ColdFusion
Posted by Steven Erat on Aug 28, 2007 at 11:48 AM
@Steven,
Thank you for the insight. To me (Based on your explanation), I feel like the "spoofing" should happen as high up in the chain as possible. From that, I would think I should choose Header over CGI in CFHttpParam whenever possible. How does that sound?
Posted by Ben Nadel on Aug 28, 2007 at 12:16 PM
This doesn't work anymore... at least not with Playboy.
The images appear to download OK, but if you open up the images, they are all corrupted files.
I guess they pulled one over your eyes? ;-)
Anthony
Posted by Anthony on Feb 1, 2008 at 5:19 PM
I try to do this with the CGI.REMOTE_ADDR but it doesn't work.
If I do this:
<cfhttp
method="post"
url="http://mytestdomain.com/iptest.cfm"
result="result">
<cfhttpparam
type="CGI"
name="remote_addr"
value="92.63.154.23"
encoded="false">
</cfhttp>
<cfoutput>#result.FileContent#</cfoutput>
and iptest.cfm contains this:
Hello from <cfoutput>#CGI.SCRIPT_NAME#</cfoutput>!<br />
Your remote_addr is <cfoutput>#CGI.remote_addr#</cfoutput>
it still outputs
Hello from /iptest.cfm!
Your remote_addr is 127.0.0.1
instead of what I expected:
Hello from /iptest.cfm!
Your remote_addr is 92.63.154.23
same with http_user_agent. I cannot overwrite this value with cfhttpparam.
Am I missing something?
Marc
Posted by Marc on Apr 8, 2008 at 5:57 AM
Try using TYPE="HEADER" instead of TYPE="CGI".
Posted by Donnie Bachan on Apr 8, 2008 at 6:15 AM
@Marc,
From what I have been told, a lot goes into calculating someone's IP address. You can't just fake it with a header or CGI value. I ran into this when I was messing with http://www.moanmyip.com . I kept trying to get her to moan numbers at my command, but alas, I could not come up with any way to fake it.
Posted by Ben Nadel on Apr 8, 2008 at 7:26 AM
The difference between type="CGI" and type="HEADER" is that the former gets urlencoded by default, the latter not. If I add encoded="false" to type="CGI" the two are identical.
To get an overview of CGI variables that you can modify and CGI variables that are not modifyable do this:
in test1.cfm:
<cfhttp method="get" url="http:mydomain.com/iptest.cfm" result="result">
<cfloop collection="#CGI#" item="CGIKey">
<cfhttpparam type="CGI" name="#CGIKey#" value="[#CGI['#CGIKey#']#] IS_MODIFYABLE" encoded="false">
</cfloop>
</cfhttp>
<cfoutput>#result.FileContent#</cfoutput>
In iptest.cfm:
<cfdump var="#CGI#">
You get a dump of the CGI scope. There are many CGI variables you can modify but remote_addr is not one of them. Actually 10 out of 44 are not modifyable on my server - Apache 2.059. Not quite clear why some values can be overwritten and others not.
Posted by Marc on Apr 8, 2008 at 7:38 AM
Hey! I used to work for her! She was a hotty well into her sixties too! She had a big set of knockers .. too bad she was a boozer. Still ... a luscious GILF after all these years!
Posted by Some Guy on Apr 15, 2008 at 3:53 PM
