Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
I am the chief technical officer at InVision App, Inc - a prototyping and collaboration platform for designers, built by designers. I also rock out in JavaScript and ColdFusion 24x7.
Meanwhile on Twitter
Loading latest tweet...
Ben Nadel at RIA Unleashed (Nov. 2009) with:

Updated CFHttpSession.cfc With Spoofed Referer Can Log Into PayPal

By Ben Nadel on
Tags: ColdFusion

Yesterday, Steve Stout brought it to my attention that my CFHttpSession.cfc ColdFusion component does not use any Referer spoofing. When I read that, I almost couldn't believe my eyes; I have spent a good amount of my time playing around with CFHttp and CFHttpParam and the fact that I forgot to put in referral spoofing blows my mind a little bit. So, this morning, I went back in and added the functionality. When you launch a new request with a given URL, you have the option to pass in a referer spoof as the second argument:

CFHttpSession.NewRequest( URL [, Referer ] )

But, I doubt that you will ever need it since I have also added in automatic referer usage based on previous URLs in the request flow. Everytime you make a new request, it checks to see if you have made a previous request with the same CFHttpSession.cfc instance. If you have, it uses the previous target URL and the current referer. This way, your page requests are really mimicing the action of a real browser.

In order to mimic a real browser, you have to think like a real user. How would a real user work? A real user wouldn't just jump directly to a form processing page. No, a real user would go to the form page first, then submit the form. This is the same mentality you have to use when using the CFHttpSession.cfc ColdFusion component; when you want to log into a site, you have to view the login page first, then submit the form with a subsequent request. This way, you can set up all the session cookies and live within the session rules set forth by the site for real live users.

Steve Stout also pointed out that he was getting connection issues when connecting to a HTTPS page on pb.com (which is actually how the discussion of referral spoofing came up). I don't have a login to pb.com, but I do have a login for PayPal.com which also uses HTTPS for their secure pages (as do most all secure pages). So, I ran some tests on logging into PayPal.com to make sure that the referral spoofing was working properly:

  • <!---
  • Create the CFHttpSession object that will be sued to
  • make our multiple calls to the same remote application.
  • --->
  • <cfset objHttpSession = CreateObject(
  • "component",
  • "CFHTTPSession"
  • ).Init()
  • />
  •  
  •  
  • <!---
  • Call the PayPal website. We need to call the homepage
  • first to set up the proper cookies and referer. If we
  • try to access the login page directly, we will get
  • denies access.
  • --->
  • <cfset objResponse = objHttpSession
  • .NewRequest( "http://www.paypal.com" )
  • .Get()
  • />
  •  
  •  
  • <!---
  • Now that we have our session set up, let's go ahead
  • and log into PayPal.com.
  • --->
  • <cfset objResponse = objHttpSession
  • .NewRequest( "https://www.paypal.com/us/cgi-bin/webscr" )
  • .AddUrl( "cmd", "_login-submit" )
  • .AddFormField( "login_email", "fergie@blackeyedpeas.com" )
  • .AddFormField( "login_password", "myLadyLumps" )
  • .AddFormField( "submit.x", "Log In" )
  • .AddFormField( "form_charset", "UTF-8" )
  • .Post()
  • />
  •  
  •  
  • <!---
  • At this point, we should be logged into the system. Now,
  • let's hop over to the account overview page.
  • --->
  • <cfset objResponse = objHttpSession
  • .NewRequest( "https://www.paypal.com/us/cgi-bin/webscr" )
  • .AddUrl( "cmd", "_account" )
  • .AddUrl( "nav", "0" )
  • .Get()
  • />
  •  
  •  
  • <!---
  • To make sure that everything is working, output the
  • PayPal account overview page.
  • --->
  • <cfoutput>
  •  
  • <h1>
  • From My Server:
  • </h1>
  •  
  • <br />
  •  
  • <div style="width: 500px ; height: 400 ; border: 4px solid gold ; overflow: auto ;">
  • #objResponse.FileContent#
  • </div>
  •  
  • </cfoutput>

Notice that we are taking three steps to get to the Account Overview page:

  1. Go to homepage (login form)
  2. Submit login credentials
  3. Go to account overview page

Using this page flow, we mimic a real user and build a proper sessoin. And, running the code above, we get the following output:


 
 
 

 
CFHttpSession.cfc Uses ColdFusion To Log Into PayPal.com  
 
 
 

As you can see, using ColdFusion and CFHttpSession.cfc, I was able to successfully log into my PayPal.com account and then further access my Account Overview page.

Here is the code for the updated CFHttpSessoin.cfc with referral spoofing (and a few other updates here and there):

  • <cfcomponent
  • output="false"
  • hint="Handles a CFHTTP session by sending an receving cookies behind the scenes.">
  •  
  • <!---
  • Pseudo constructor. Set up data structures and
  • default values.
  • --->
  • <cfset VARIABLES.Instance = {} />
  •  
  • <!---
  • These are the cookies that get returned from the
  • request that enable us to keep the session across
  • different CFHttp requests.
  • --->
  • <cfset VARIABLES.Instance.Cookies = {} />
  •  
  • <!---
  • The request data contains the various types of data that
  • we will send with our request. These will be both for the
  • CFHttpParam tags as well as the CFHttp property values.
  • --->
  • <cfset VARIABLES.Instance.RequestData = {} />
  • <cfset VARIABLES.Instance.RequestData.Url = "" />
  • <cfset VARIABLES.Instance.RequestData.Referer = "" />
  • <cfset VARIABLES.Instance.RequestData.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6" />
  • <cfset VARIABLES.Instance.RequestData.Params = [] />
  •  
  •  
  • <cffunction
  • name="Init"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Returns an initialized component.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="UserAgent"
  • type="string"
  • required="false"
  • hint="The user agent that will be used on the subseqent page requests."
  • />
  •  
  • <!--- Check to see if we have a user agent. --->
  • <cfif StructKeyExists( ARGUMENTS, "UserAgent" )>
  • <cfset THIS.SetUserAgent( ARGUMENTS.UserAgent ) />
  • </cfif>
  •  
  • <!--- Return This reference. --->
  • <cfreturn THIS />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddCGI"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a CGI value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the CGI value."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="string"
  • required="true"
  • hint="The CGI value."
  • />
  •  
  • <cfargument
  • name="Encoded"
  • type="string"
  • required="false"
  • default="yes"
  • hint="Determins whether or not to encode the CGI value."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "CGI",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value,
  • Encoded = ARGUMENTS.Encoded
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddCookie"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a cookie value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the CGI value."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="string"
  • required="true"
  • hint="The CGI value."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Cookie",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddFile"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a file value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the form field for the posted file."
  • />
  •  
  • <cfargument
  • name="Path"
  • type="string"
  • required="true"
  • hint="The expanded path to the file."
  • />
  •  
  • <cfargument
  • name="MimeType"
  • type="string"
  • required="false"
  • default="application/octet-stream"
  • hint="The mime type of the posted file. Defaults to *unknown* mime type."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Cookie",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddFormField"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a form value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the form field."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="string"
  • required="true"
  • hint="The form field value."
  • />
  •  
  • <cfargument
  • name="Encoded"
  • type="string"
  • required="false"
  • default="yes"
  • hint="Determins whether or not to encode the form value."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "FormField",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value,
  • Encoded = ARGUMENTS.Encoded
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddHeader"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a header value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the header value."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="string"
  • required="true"
  • hint="The header value."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Header",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddParam"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a CFHttpParam data point. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Type"
  • type="string"
  • required="true"
  • hint="The type of data point."
  • />
  •  
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the data point."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="any"
  • required="true"
  • hint="The value of the data point."
  • />
  •  
  • <cfargument
  • name="File"
  • type="string"
  • required="false"
  • default=""
  • hint="The expanded path to be used if the data piont is a file."
  • />
  •  
  • <cfargument
  • name="MimeType"
  • type="string"
  • required="false"
  • default=""
  • hint="The mime type of the file being passed (if file is being passed)."
  • />
  •  
  • <cfargument
  • name="Encoded"
  • type="string"
  • required="false"
  • default="yes"
  • hint="The determines whether or not to encode Form Field and CGI values."
  • />
  •  
  • <!--- Define the local scope. --->
  • <cfset var LOCAL = {} />
  •  
  • <!---
  • Check to see which kind of data point we are dealing
  • with so that we can see how to create the param.
  • --->
  • <cfswitch expression="#ARGUMENTS.Type#">
  •  
  • <cfcase value="Body">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Value = ARGUMENTS.Value
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="CGI">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value,
  • Encoded = ARGUMENTS.Encoded
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="Cookie">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="File">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • File = ARGUMENTS.File,
  • MimeType = ARGUMENTS.MimeType
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="FormField">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value,
  • Encoded = ARGUMENTS.Encoded
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="Header">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="Url">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • } />
  •  
  • </cfcase>
  •  
  • <cfcase value="Xml">
  •  
  • <!--- Create the param. --->
  • <cfset LOCAL.Param = {
  • Type = ARGUMENTS.Type,
  • Value = ARGUMENTS.Value
  • } />
  •  
  • </cfcase>
  •  
  • </cfswitch>
  •  
  •  
  • <!--- Add the parameter for the next request. --->
  • <cfset ArrayAppend(
  • VARIABLES.Instance.RequestData.Params,
  • LOCAL.Param
  • ) />
  •  
  • <!--- Return This reference. --->
  • <cfreturn THIS />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="AddUrl"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Adds a url value. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Name"
  • type="string"
  • required="true"
  • hint="The name of the header value."
  • />
  •  
  • <cfargument
  • name="Value"
  • type="string"
  • required="true"
  • hint="The header value."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Url",
  • Name = ARGUMENTS.Name,
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="Get"
  • access="public"
  • returntype="struct"
  • output="false"
  • hint="Uses the GET method to place the next request. Returns the CFHttp response.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="GetAsBinary"
  • type="string"
  • required="false"
  • default="auto"
  • hint="Determines how to return the file content - return as binary value."
  • />
  •  
  • <cfargument
  • name="Debug"
  • type="boolean"
  • required="false"
  • default="false"
  • hint="If this is true, then the response object will be dumped out and the page will be aborted."
  • />
  •  
  • <!--- Return response. --->
  • <cfreturn THIS.Request(
  • Method = "get",
  • GetAsBinary = ARGUMENTS.GetAsBinary,
  • Debug = ARGUMENTS.Debug
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="GetCookies"
  • access="public"
  • returntype="struct"
  • output="false"
  • hint="Returns the internal session cookies.">
  •  
  • <cfreturn VARIABLES.Instance.Cookies />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="NewRequest"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Sets up the object for a new request. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Url"
  • type="string"
  • required="true"
  • hint="The URL for the new request."
  • />
  •  
  • <cfargument
  • name="Referer"
  • type="string"
  • required="false"
  • default=""
  • hint="The referring URL for the request. By default, it will be the same directory as the target URL."
  • />
  •  
  •  
  • <!---
  • Before we store the URL, let's check to see if we
  • already had one in memory. If so, then we can use
  • that for a referer (which we then have the option
  • to override. The point here is that each URL can
  • be the referer for the next one.
  • --->
  • <cfif Len( VARIABLES.Instance.RequestData.Url )>
  •  
  • <!---
  • Store the previous url as the next referer. We
  • may override this in a second.
  • --->
  • <cfset VARIABLES.Instance.RequestData.Referer = VARIABLES.Instance.RequestData.Url />
  •  
  • </cfif>
  •  
  •  
  • <!--- Store the passed-in url. --->
  • <cfset VARIABLES.Instance.RequestData.Url = ARGUMENTS.Url />
  •  
  • <!---
  • Check to see if the referer was passed in. Since we
  • are using previous URLs as the next referring url,
  • we only want to store the passed in value if it has
  • length
  • --->
  • <cfif Len( ARGUMENTS.Referer )>
  •  
  • <!--- Store manually set referer. --->
  • <cfset VARIABLES.Instance.RequestData.Referer = ARGUMENTS.Referer />
  •  
  • </cfif>
  •  
  •  
  • <!--- Clear the request data. --->
  • <cfset VARIABLES.Instance.RequestData.Params = [] />
  •  
  • <!--- Return This reference. --->
  • <cfreturn THIS />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="Post"
  • access="public"
  • returntype="struct"
  • output="false"
  • hint="Uses the POST method to place the next request. Returns the CFHttp response.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="GetAsBinary"
  • type="string"
  • required="false"
  • default="auto"
  • hint="Determines how to return the file content - return as binary value."
  • />
  •  
  • <cfargument
  • name="Debug"
  • type="boolean"
  • required="false"
  • default="false"
  • hint="If this is true, then the response object will be dumped out and the page will be aborted."
  • />
  •  
  • <!--- Return response. --->
  • <cfreturn THIS.Request(
  • Method = "post",
  • GetAsBinary = ARGUMENTS.GetAsBinary,
  • Debug = ARGUMENTS.Debug
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="Request"
  • access="public"
  • returntype="struct"
  • output="false"
  • hint="Performs the CFHttp request and returns the response.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Method"
  • type="string"
  • required="false"
  • default="get"
  • hint="The type of request to make."
  • />
  •  
  • <cfargument
  • name="GetAsBinary"
  • type="string"
  • required="false"
  • default="auto"
  • hint="Determines how to return body."
  • />
  •  
  • <cfargument
  • name="Debug"
  • type="boolean"
  • required="false"
  • default="false"
  • hint="If this is true, then the response object will be dumped out and the page will be aborted."
  • />
  •  
  • <!--- Define the local scope. --->
  • <cfset var LOCAL = {} />
  •  
  • <!---
  • Make request. When the request comes back, we don't
  • want to follow any redirects. We want this to be
  • done manually.
  • --->
  • <cfhttp
  • url="#VARIABLES.Instance.RequestData.Url#"
  • method="#ARGUMENTS.Method#"
  • useragent="#VARIABLES.Instance.RequestData.UserAgent#"
  • getasbinary="#ARGUMENTS.GetAsBinary#"
  • redirect="no"
  • result="LOCAL.Get">
  •  
  • <!---
  • In order to maintain the user's session, we are
  • going to resend any cookies that we have stored
  • internally.
  • --->
  • <cfloop
  • item="LOCAL.Key"
  • collection="#VARIABLES.Instance.Cookies#">
  •  
  • <cfhttpparam
  • type="cookie"
  • name="#LOCAL.Key#"
  • value="#VARIABLES.Instance.Cookies[ LOCAL.Key ].Value#"
  • />
  •  
  • </cfloop>
  •  
  •  
  • <!---
  • At this point, we have done everything that we
  • need to in order to maintain the user's session
  • across CFHttp requests. Now we can go ahead and
  • pass along any addional data that has been specified.
  • --->
  •  
  •  
  • <!--- Let's spoof the referer. --->
  • <cfhttpparam
  • type="header"
  • name="referer"
  • value="#VARIABLES.Instance.RequestData.Referer#"
  • />
  •  
  •  
  • <!--- Loop over params. --->
  • <cfloop
  • index="LOCAL.Param"
  • array="#VARIABLES.Instance.RequestData.Params#">
  •  
  • <!---
  • Pass the existing param object in as our
  • attributes collection.
  • --->
  • <cfhttpparam
  • attributecollection="#LOCAL.Param#"
  • />
  •  
  • </cfloop>
  •  
  • </cfhttp>
  •  
  •  
  • <!---
  • Check to see if we are debugging. If so, then we
  • will dump out the request response and abort the
  • page flow.
  • --->
  • <cfif ARGUMENTS.Debug>
  •  
  • <!--- Dump and abort. --->
  • <cfdump var="#VARIABLES.Instance.RequestData#" />
  • <cfset WriteOutput( LOCAL.Get.FileContent ) />
  • <cfdump var="#LOCAL.Get#" />
  • <cfabort />
  •  
  • </cfif>
  •  
  •  
  • <!---
  • Store the response cookies into our internal cookie
  • storage struct.
  • --->
  • <cfset StoreResponseCookies( LOCAL.Get ) />
  •  
  •  
  • <!---
  • Check to see if there was some sort of redirect
  • returned with the repsonse. If there was, we want
  • to redirect with the proper value.
  • --->
  • <cfif StructKeyExists( LOCAL.Get.ResponseHeader, "Location" )>
  •  
  • <!---
  • There was a response, so now we want to do a
  • recursive call to return the next page. When
  • we do this, make sure we have the proper URL
  • going out.
  • --->
  • <cfif REFindNoCase(
  • "^http",
  • LOCAL.Get.ResponseHeader.Location
  • )>
  •  
  • <!--- Proper url. --->
  • <cfreturn THIS
  • .NewRequest( LOCAL.Get.ResponseHeader.Location )
  • .Get()
  • />
  •  
  • <cfelse>
  •  
  • <!---
  • Non-root url. We need to append the current
  • redirect url to our last URL for relative
  • path traversal.
  • --->
  • <cfreturn THIS
  • .NewRequest(
  • GetDirectoryFromPath( VARIABLES.Instance.RequestData.Url ) &
  • LOCAL.Get.ResponseHeader.Location
  • )
  • .Get()
  • />
  •  
  • </cfif>
  •  
  • <cfelse>
  •  
  • <!---
  • No redirect, so just return the current
  • request response object.
  • --->
  • <cfreturn LOCAL.Get />
  •  
  • </cfif>
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="SetBody"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Sets the body data of next request. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Value"
  • type="any"
  • required="false"
  • hint="The data body."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Body",
  • Name = "",
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="SetUserAgent"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Sets the user agent for next request. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Value"
  • type="string"
  • required="false"
  • hint="The user agent that will be used on the subseqent page requests."
  • />
  •  
  • <!--- Store value. --->
  • <cfset VARIABLES.Instance.RequestData.UserAgent = ARGUMENTS.UserAgent />
  •  
  • <!--- Return This reference. --->
  • <cfreturn THIS />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="SetXml"
  • access="public"
  • returntype="any"
  • output="false"
  • hint="Sets the XML body data of next request. Returns THIS scope for method chaining.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Value"
  • type="any"
  • required="false"
  • hint="The data body."
  • />
  •  
  • <!--- Set parameter and return This reference. --->
  • <cfreturn THIS.AddParam(
  • Type = "Xml",
  • Name = "",
  • Value = ARGUMENTS.Value
  • ) />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="StoreResponseCookies"
  • access="public"
  • returntype="void"
  • output="false"
  • hint="This parses the response of a CFHttp call and puts the cookies into a struct.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="Response"
  • type="struct"
  • required="true"
  • hint="The response of a CFHttp call."
  • />
  •  
  • <!--- Define the local scope. --->
  • <cfset var LOCAL = StructNew() />
  •  
  • <!---
  • Create the default struct in which we will hold
  • the response cookies. This struct will contain structs
  • and will be keyed on the name of the cookie to be set.
  • --->
  • <cfset LOCAL.Cookies = StructNew() />
  •  
  • <!---
  • Get a reference to the cookies that werew returned
  • from the page request. This will give us an numericly
  • indexed struct of cookie strings (which we will have
  • to parse out for values). BUT, check to make sure
  • that cookies were even sent in the response. If they
  • were not, then there is not work to be done.
  • --->
  • <cfif NOT StructKeyExists(
  • ARGUMENTS.Response.ResponseHeader,
  • "Set-Cookie"
  • )>
  •  
  • <!--- No cookies were send back so just return. --->
  • <cfreturn />
  •  
  • </cfif>
  •  
  •  
  • <!---
  • ASSERT: We know that cookie were returned in the page
  • response and that they are available at the key,
  • "Set-Cookie" of the reponse header.
  • --->
  •  
  •  
  • <!---
  • The cookies might be coming back as a struct or they
  • might be coming back as a string. If there is only
  • ONE cookie being retunred, then it comes back as a
  • string. If that is the case, then re-store it as a
  • struct.
  • --->
  • <cfif IsSimpleValue( ARGUMENTS.Response.ResponseHeader[ "Set-Cookie" ] )>
  •  
  • <cfset LOCAL.ReturnedCookies = {} />
  • <cfset LOCAL.ReturnedCookies[ 1 ] = ARGUMENTS.Response.ResponseHeader[ "Set-Cookie" ] />
  •  
  • <cfelse>
  •  
  • <!--- Get a reference to the cookies struct. --->
  • <cfset LOCAL.ReturnedCookies = ARGUMENTS.Response.ResponseHeader[ "Set-Cookie" ] />
  •  
  • </cfif>
  •  
  •  
  • <!---
  • At this point, we know that no matter how the
  • cookies came back, we have the cookies in a
  • structure of cookie values.
  • --->
  • <cfloop
  • item="LOCAL.CookieIndex"
  • collection="#LOCAL.ReturnedCookies#">
  •  
  • <!---
  • As we loop through the cookie struct, get
  • the cookie string we want to parse.
  • --->
  • <cfset LOCAL.CookieString = LOCAL.ReturnedCookies[ LOCAL.CookieIndex ] />
  •  
  •  
  • <!---
  • For each of these cookie strings, we are going
  • to need to parse out the values. We can treate
  • the cookie string as a semi-colon delimited list.
  • --->
  • <cfloop
  • index="LOCAL.Index"
  • from="1"
  • to="#ListLen( LOCAL.CookieString, ';' )#"
  • step="1">
  •  
  • <!--- Get the name-value pair. --->
  • <cfset LOCAL.Pair = ListGetAt(
  • LOCAL.CookieString,
  • LOCAL.Index,
  • ";"
  • ) />
  •  
  • <!---
  • Get the name as the first part of the pair
  • sepparated by the equals sign.
  • --->
  • <cfset LOCAL.Name = ListFirst( LOCAL.Pair, "=" ) />
  •  
  • <!---
  • Check to see if we have a value part. Not all
  • cookies are going to send values of length,
  • which can throw off ColdFusion.
  • --->
  • <cfif (ListLen( LOCAL.Pair, "=" ) GT 1)>
  •  
  • <!--- Grab the rest of the list. --->
  • <cfset LOCAL.Value = ListRest( LOCAL.Pair, "=" ) />
  •  
  • <cfelse>
  •  
  • <!---
  • Since ColdFusion did not find more than
  • one value in the list, just get the empty
  • string as the value.
  • --->
  • <cfset LOCAL.Value = "" />
  •  
  • </cfif>
  •  
  •  
  • <!---
  • Now that we have the name-value data values,
  • we have to store them in the struct. If we
  • are looking at the first part of the cookie
  • string, this is going to be the name of the
  • cookie and it's struct index.
  • --->
  • <cfif (LOCAL.Index EQ 1)>
  •  
  • <!---
  • Create a new struct with this cookie's name
  • as the key in the return cookie struct.
  • --->
  • <cfset LOCAL.Cookies[ LOCAL.Name ] = StructNew() />
  •  
  • <!---
  • Now that we have the struct in place, lets
  • get a reference to it so that we can refer
  • to it in subseqent loops.
  • --->
  • <cfset LOCAL.Cookie = LOCAL.Cookies[ LOCAL.Name ] />
  •  
  •  
  • <!--- Store the value of this cookie. --->
  • <cfset LOCAL.Cookie.Value = LOCAL.Value />
  •  
  •  
  • <!---
  • Now, this cookie might have more than just
  • the first name-value pair. Let's create an
  • additional attributes struct to hold those
  • values.
  • --->
  • <cfset LOCAL.Cookie.Attributes = StructNew() />
  •  
  • <cfelse>
  •  
  • <!---
  • For all subseqent calls, just store the
  • name-value pair into the established
  • cookie's attributes strcut.
  • --->
  • <cfset LOCAL.Cookie.Attributes[ LOCAL.Name ] = LOCAL.Value />
  •  
  • </cfif>
  •  
  • </cfloop>
  •  
  •  
  • </cfloop>
  •  
  •  
  • <!---
  • Now that we have all the response cookies in a
  • struct, let's append those cookies to our internal
  • response cookies.
  • --->
  • <cfset StructAppend(
  • VARIABLES.Instance.Cookies,
  • LOCAL.Cookies
  • ) />
  •  
  • <!--- Return out. --->
  • <cfreturn />
  • </cffunction>
  •  
  • </cfcomponent>



Reader Comments

Ben,
As always, this is incredible. This is hopefully going to completely solve a problem we've had with automating deployment of our sites, which have to go through a 3rd party CMS system.

Anyway, I've run into an issue, where the redirect due to a Location caused a problem when the redirection wasn't a full redirect with http:// but instead an absolute redirect. It appends that new location onto the full request URL, which means that you get a 404.

Maybe an example is easiest.

http://apple.com/foo

302 Moved Temporarily
Location: /bar

With the way it's currently programmed, the new request goes to http://apple.com/foo/bar, and you get a 404.

I made a change to the component that seems to work on my end, but I'm sure I'm missing something. Let me know if you want a copy of the changed code.

Thanks again for this great tool!

@Toby,

The CFHttpSession.cfc component, when it finds a locational redirect is checking to see if the URL passed back starts with "http:". If it does, then is uses the given URL. If it does NOT, then it takes the redirect and appends it to fileless-path of the previous URL.

Are you using CFHttpSession.cfc for your stuff? If so, please let me know of any problems it is causing.

@Ben,
I guess that's my point it's NOT removing the path at the end of the URL. It's possible that the behavior is different on PC based systems, but on my sysem, if you send in the URL http://apple.com/foo but there's a location redirect to /bar, the component doesn't request http://apple.com/bar but instead asks for http://apple.com/foo/bar. This might be related to the GetDirectoryFromPath() function working differently on PC vs. *nix based systems (I'm using OS X).

I haven't started actively using the component yet -- I was just testing it out to see if it could fulfill a need we have. I haven't got it to work 100% yet (need to work out a few kinks on my end).

Thanks again,
Toby

Just to be clear, I'm talking about an instance where /foo and /bar are both directory paths (i.e. it could be /foo/, /bar/, but the system I connect to is just returning /bar).

I can show you the specific part of the code that's affected, if you'd like.

I missed your original blog post "Passing Referer AS ColdFusion CFHttp CGI Value vs HEADER Value" from last August. That was damned clever of you, Ben!

Ironically, a new spam comment was added to that very post a few weeks ago. At least, I think it's spam. If not, you have a seriously tweaked fan base. :)

@Toby,

That's odd. The GetDirectoryFromPath() should strip off "foo" from ..../foo since Foo does not have a trailing slash. I will look into it some more.

@Ben,
Hmmm....well, the example I gave maybe wasn't great. Let me use something closer to reality.

Let's say the URL was:

http://domain.com/do/action

And then it had a 302 redirect to /do/otheraction

Right now, your code would redirect it to /do/action/do/otheraction.

BUT even if you stripped off the first unslashed entry from the original URL, so you got /do/do/otheraction, you'd still get a 404.

My guess is that you have to strip off the whole path. Does that not match the specification for HTTP 302 redirects?

Thanks,
Toby

@Toby,

Ahhh, thanks. I see what the problem is. I can't just check for http:, I also have to check for absolute URLs, "/" as well as relative "../". This will be easy to fix. I will do that when I get back from cf.Objective(). Thanks.

Hi Ben,

I see, from your screenshot, that CFHTTPSession.cfc has worked against PayPal before. Apparently PayPal did not previously send cookies having special characters. Currently, PayPal is sending a cookie containing a pipe-delimited list, and another cookie having an ampersand-delimited list of name/value pairs. Thus, the pipe, ampersand, and equal sign are received as encoded characters. Running the cookies containing these new already-encoded special characters thru urlEncodedFormat, and passing them back to PayPal, prevents successful login attempt. Thus, the fix is to decode the received cookies before encoding them for sending. On line 1140 of 2010/02/05 build of CFHTTPSession.cfc (where it parses the returned cookie value), I've wrapped the value w/ urlDecode.

Thus, ListRest( LOCAL.Pair, "=" ) becomes UrlDecode(ListRest( LOCAL.Pair, "=" )).

I want to thank you very much for all the time you've invested into CFHTTPSession.cfc. Perhaps <cfhttp /> can be enhanced as thus: <cfhttp session="mySessionName" />, to allow session to be maintained amongst calls having same-named session attribute (similar to cfx_http5).

FYI, I commented out the referer spoof, and call was still successful. It seems PayPal is not (or no longer) relying on valid referer.

Thanks again!,
-Aaron Neff

@Aaron,

Very awesome detective work. I recently tried to get PayPal integration to work and was unable to get this to happen (even after starting from scratch). I'll definitely be testing this tomorrow!

@Ben,

Thanks! I thought I was losin' it..

So I did some googling and came across cfx_http5. I pretty much just implemented the code right out of their example, and was able to maintain a PayPal session just fine. So I started comparing the communication w/ Firebug and Wireshark, and also had the form post to another script on my server (instead of just PayPal). I noticed that if I pass the cookies via <cfhttpparam type="header" name="Cookie" value="semicolon delimited list of cookie name/value pairs here" /> (along w/ another header to set the content type as application/x-www-form-urlencoded), that I was able to maintain a session.

So I was almost ready to file an ER to allow cfhttpparam type="cookie" to support 'encoded="false"', but then I realized that we really should probably be unencoding the cookies, before we attempt to encode them again (which is what cfhttpparam/cookie does behind-the-scenes).

Would've been nicer if wireshark would decode the https communication w/ PayPal, but that's ok.. we got it figured out! (or I hope so.. just awaiting your confirmation)

I like that CFHTTPSession.cfc :) You really thought that out! Now to get Adobe to upgrade cfhttp to support session sharing btwn calls :)

Thanks!,
-Aaron

@Aaron,

Awesome stuff (sorry for the super late reply - had this tab open for weeks). I keep trying to remember to update my CFHTTPSession.cfc project based on your findings. Very awesome!

@Ben,

And here I am w/ another super late reply! haha, I just saw your message here from last month. Cool, the only thing I added was the urlDecode() on line 1140. Thank you, and you're welcome, and I'll be interested to know when the updated CFHTTPSession.cfc is posted.

Take care!,
-Aaron

@Aaron,

No problem at all on the late reply - I still haven't updated the project page ;) We're all busy - no judgements!

I just tried this paypal login on a openbluedragon 1.4a server and got this error:

Detail Missing Attribute: This tag requires a TYPE attribute

^ Snippet from underlying CFML source
Type Template
Message Invalid Attribute
Tag CFHTTPPARAM
Position Line=855; Column=8
Detail Missing Attribute: This tag requires a TYPE attribute
Source

852: --->
853: <cfhttpparam
854: attributecollection="#LOCAL.Param#"
855: />
856:

openBD 1.4 has support for implicit structures and arrays and also attributecollections. So I am stuck right now it should have worked. I will try some basic page scraping to see if it will work.

Great it works, at least I got it working on CF9 developer version on ubuntu after adding URLDEcode on line 1140 and changing some stuff to swedish (for the swedish site paypal.com/se).