Streaming Secure Files Efficiently With ColdFusion And MOD XSendFile

Posted April 14, 2011 at 11:12 AM by Ben Nadel

Tags: ColdFusion

ColdFusion is truly the cat's pajamas. But, it only works on CFM and CFC pages. This typically isn't a problem; but, when it comes to adding access-permissions around non-ColdFusion files, we can find ourselves in a less-than-optimal situation. While it has been shown that ColdFusion can stream binary files without a significant memory hit, we still have the issue of threads - streaming a file using ColdFusion and the CFContent tag ties up a ColdFusion thread for the duration of the file transfer. Ideally, we'd like to be able to perform a security check in ColdFusion and then pass the file-transfer burden off to the underlying operating system. Luckily, MOD XSendFile does exactly this.

 
 
 
 
 
 
 
 
 
 

XSendFile is an Apache module that allows ColdFusion (and other application platforms) to delegate file transfer actions back to Apache. It does this through the use of an HTTP Header. After ColdFusion has processed an incoming request, the XSendFile module will examine the outgoing HTTP headers; if the "x-sendfile" header exists, Apache overwrites the outgoing request with the file defined in the "x-sendfile" header.

NOTE: While XSendFile is an Apache module, there are ways that you can run Apache configurations in IIS (ex. Helicon Ape).

To demonstrate how MOD XSendFile works, I've set up a super simple, mock store on my local Apache server. In this store, users can exchange credits in return for access to secure photos. These photos reside outside the web root; as such, we need to use ColdFusion to determine access rights on a per-photo basis and then to pass the non-public photo paths back to Apache for binary streaming.

To get this to work, I had to install the XSendFile module in Apache, load the module, and then enable it for one of my virtual hosts:

  • ## NOTE: The installation automatically added this module loader
  • ## into my main httpd.conf file (not shown in the demo).
  • ##
  • ## LoadModule xsendfile_module libexec/apache2/mod_xsendfile.so
  •  
  •  
  • <VirtualHost *:80>
  •  
  • # Define our domain name for local routing.
  • ServerName xsendfile.com
  •  
  • # Define the webroot for the site.
  • DocumentRoot "/Sites/bennadel.com/testing/mod_xsendfile/www"
  •  
  • # Include the ColdFusion connector.
  • Include "/private/etc/apache2/cf8-main.conf"
  •  
  • # Set permissions for web root directory.
  • <Directory "/Sites/bennadel.com/testing/mod_xsendfile/www">
  •  
  • # Enable the use of MOD_XSendFile.
  •  
  • XSendFile On
  •  
  • # Allow MOD_XSendFile to access files in the directory
  • # parallel to the web root (images). If we don't set this,
  • # the MOD_XSendFile will only be able to access files in
  • # or below the defined web root.
  • #
  • # NOTE: This used to be the property, XSendFileAllowAbove,
  • # but it has been deprecated in newer versions of the
  • # XSendFile module.
  •  
  • XSendFilePath "/Sites/bennadel.com/testing/mod_xsendfile/images"
  •  
  • </Directory>
  •  
  • </VirtualHost>

Once the XSendFile module is installed, it has to be explicitly enabled. In the above Apache configuration, the following command turns XSendFile on for a given directory:

  • XSendFile On

This will allow XSendFile to access files anywhere in or below the contextual directory. The problem with that is that I am defining this within my web root; and, for obvious security purposes, I am storing the photos outside of the web root. As such, I then have to specify that XSendFile can access files that reside outside of the web root; specifically, that it can access files in my private "images" directory:

  • XSendFilePath "/Sites/bennadel.com/testing/mod_xsendfile/images"

NOTE: In previous versions of XSendFile, this same action used to be accomplished with XSendFileAllowAbove. This directive no longer exists and has been replaced with XSendFilePath.

In the end, this is what my directory structure looked like:

  • ./site/www/
  • ./site/www/thumbs/
  • ./site/images/

As you can see, the "www" is our public website. Notice that the thumbnails are stored in a public, web-accessible directory - thumbs. There's no need to secure those. The large photos, on the other hand, are our primary assets and are stored outside the web root in the parallel, "images," directory.

Now that we have XSendFile installed and configured, let's take a look at the ColdFusion code. I'm going to move through this code rather quickly since 90% of it isn't really relative to the XSendFile Apache module - it's just there to provide context.

Application.cfc

  • <cfcomponent
  • output="false"
  • hint="I define the application settings and event handlers.">
  •  
  • <!--- Define the application settings. --->
  • <cfset this.name = hash( getCurrentTemplatePath() ) />
  • <cfset this.applicationTimeout = createTimeSpan( 0, 0, 10, 0 ) />
  • <cfset this.sessionManagement = true />
  • <cfset this.sessionTimeout = createTimeSpan( 0, 0, 10, 0 ) />
  •  
  •  
  • <cffunction
  • name="onApplicationStart"
  • access="public"
  • returntype="boolean"
  • output="false"
  • hint="I initialize the application.">
  •  
  • <!---
  • Get the path to the images directory. For this demo,
  • that is simply the directory paralle to the webroot
  • (ie. ../images/).
  •  
  • NOTE: For this demo, I'm going to use "../" notation for
  • path traversal rather than actually calculating the exact
  • path - just makes the demo easier to read.
  • --->
  • <cfset application.imagesDirectory = (
  • getDirectoryFromPath( getCurrentTemplatePath() ) &
  • "../images/"
  • ) />
  •  
  • <!--- Return true so the application can load. --->
  • <cfreturn true />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="onSessionStart"
  • access="public"
  • returntype="void"
  • output="false"
  • hint="I initialize the session.">
  •  
  • <!---
  • Start the user off with a balance (which can be traded
  • in for image access).
  • --->
  • <cfset session.balance = 5.0 />
  •  
  • <!---
  • Keep track of the user's purchases. When the user trades
  • in their balance for an image, we will keep a record of
  • it so they can access the given files.
  • --->
  • <cfset session.purchases = {} />
  •  
  • <!--- Return out. --->
  • <cfreturn />
  • </cffunction>
  •  
  •  
  • <cffunction
  • name="onRequestStart"
  • access="public"
  • returntype="boolean"
  • output="false"
  • hint="I initialize the request.">
  •  
  • <!--- Check for a re-init flag. --->
  • <cfif structKeyExists( url, "init" )>
  •  
  • <!--- Manually reset the application. --->
  • <cfset this.onApplicationStart() />
  • <cfset this.onSessionStart() />
  •  
  • </cfif>
  •  
  • <!--- Return true so the request can load. --->
  • <cfreturn true />
  • </cffunction>
  •  
  • </cfcomponent>

When a user's session gets initialized, they get two properties - balance and purchases. The balance is their store credit and the purchases is an ID-based collection of photos for which they have paid.

The main page of the application handles the photo-list and purchase-processing. The user selects a photo and, if they have enough credits, the photo is added to their list of purchases and the user is forwarded to the purchase page.

Index.cfm

  • <!--- Param the photo selection. --->
  • <cfparam name="url.selectedPhoto" type="numeric" default="0" />
  •  
  • <!--- Set an error message. --->
  • <cfset errorMessage = "" />
  •  
  • <!--- Set the cost of the photo purhcases. --->
  • <cfset photoCost = 2 />
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!--- Check to see if any photo selection was made. --->
  • <cfif url.selectedPhoto>
  •  
  • <!---
  • Check to see if the user has enough credits to make the
  • given photo purchase (or if they have already purchased
  • this photo - in which case, we'll just forward them to
  • their purchased photo).
  • --->
  • <cfif (
  • (session.balance gte photoCost) ||
  • structKeyExists( session.purchases, url.selectedPhoto )
  • )>
  •  
  • <!---
  • For this demo, we don't want to re-charge the user for
  • photos they've already purchased. As such, only charge
  • them if the photo is not already in the collection of
  • purchased items.
  • --->
  • <cfif !structKeyExists( session.purchases, url.selectedPhoto )>
  •  
  • <!--- Deduct the credit from the balance. --->
  • <cfset session.balance -= photoCost />
  •  
  • <!--- Add the purchase record. --->
  • <cfset session.purchases[ selectedPhoto ] = now() />
  •  
  • </cfif>
  •  
  • <!---
  • Forward the user to purchase page where they can view
  • and download the purchased photo.
  • --->
  • <cflocation
  • url="./purchase.cfm?photo=#url.selectedPhoto#"
  • addtoken="false"
  • />
  •  
  • <cfelse>
  •  
  • <!---
  • The use doesn't have enough balance to make the purchase
  • - set an error message to display.
  • --->
  • <cfset errorMessage = "You do not have enough credits." />
  •  
  • </cfif>
  •  
  • </cfif>
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <cfoutput>
  •  
  • <!DOCTYPE html>
  • <html>
  • <head>
  • <title>Using MOD_XSendFile With ColdFusion To Stream Files</title>
  •  
  • <style type="text/css">
  •  
  • img {
  • border: 5px solid ##454545 ;
  • margin-right: 10px ;
  • vertical-align: middle ;
  • }
  •  
  • button {
  • height: 40px ;
  • }
  •  
  • p.error {
  • color: ##CC0000 ;
  • font-style: italic ;
  • }
  •  
  • </style>
  • </head>
  • <body>
  •  
  • <h1>
  • Using MOD_XSendFile With ColdFusion To Stream Files
  • </h1>
  •  
  • <p>
  • Credit Balance: <strong>#session.balance#</strong>
  • </p>
  •  
  • <!--- Check to see if there is any error to report. --->
  • <cfif len( errorMessage )>
  •  
  • <p class="error">
  • <strong>Oh Noes!</strong> #errorMessage#
  •  
  • <a href="#cgi.script_name#?init=1">Re-initialize!</a>
  • </p>
  •  
  • </cfif>
  •  
  • <form action="#cgi.script_name#" method="get">
  •  
  • <p>
  • Please select the image to purchase:
  • </p>
  •  
  • <!--- Output a purchase option for each image. --->
  • <cfloop
  • index="photoIndex"
  • from="1"
  • to="15"
  • step="1">
  •  
  • <!---
  • Notice that thumbnails are below the webroot
  • (can be publicly accessed) but that the original
  • image are NOT in the webroot and therefore cannot
  • be accessed directly.
  • --->
  • <p>
  • <img src="./thumbs/#photoIndex#.jpg" />
  •  
  • <button
  • type="submit"
  • name="selectedPhoto"
  • value="#photoIndex#">
  • Photo #photoIndex# : #photoCost# Credits
  • </button>
  • </p>
  •  
  • </cfloop>
  •  
  • </form>
  •  
  • </body>
  • </html>
  •  
  • </cfoutput>

Once the user had made the purchase of a photo, they are forwarded to the purchase page. The purchase page doesn't actually stream the binary file to the client; rather, it acts as an intermediary landing page that gives the user some informational feedback. It's one of those "your download will begin shortly" pages.

Purchase.cfm

  • <!--- Param the photo id. --->
  • <cfparam name="url.photo" type="numeric" />
  •  
  • <cfoutput>
  •  
  • <!DOCTYPE html>
  • <html>
  • <head>
  • <title>Your Download Will Begin Shortly</title>
  •  
  • <!---
  • Forward the user to the download page. This is the
  • page that will actually stream the file to client
  • (the user's browser).
  • --->
  • <meta
  • http-equiv="refresh"
  • content="1; url=./download.cfm?photo=#url.photo#"
  • />
  • </head>
  • <body>
  •  
  • <h1>
  • Thank You - Your Download Will Begin Shortly
  • </h1>
  •  
  • <p>
  • If your download does not begin within a few seconds,
  • <a href="./download.cfm?photo=#url.photo#">click here</a>
  • to proceed manually.
  • </p>
  •  
  • <p>
  • <a href="./index.cfm">Return to Photos</a>
  • </p>
  •  
  • </body>
  • </html>
  •  
  • </cfoutput>

This page then forwards the user onto the actual download page which is where we finally get to deal with code that relates to XSendFile. Notice that ColdFusion isn't actually reading in or streaming out any file.

Download.cfm

  • <!--- Param the photo id. --->
  • <cfparam name="url.photo" type="numeric" />
  •  
  •  
  • <!---
  • Check to make sure the photo is in the user's collection
  • of purchases.
  • --->
  • <cfif structKeyExists( session.purchases, url.photo )>
  •  
  •  
  • <!---
  • The user has made the purchase - add the full file path of
  • the purchased image to the outgoing headers. By using the
  • XSendFile header, the file streaming will be passed off to
  • Apache (ie. it won't tie up a ColdFusion thread).
  •  
  • NOTE: Apache will strip this header out - it will never
  • reach the client.
  • --->
  • <cfheader
  • name="x-sendfile"
  • value="#application.imagesDirectory##url.photo#.jpg"
  • />
  •  
  • <!---
  • Deliver this purchased photo as an attachment so
  • the "Purchase" screen will not be navigated away from
  • (visually speaking).
  • --->
  • <cfheader
  • name="content-disposition"
  • value="attachment; filename=purchase-#url.photo#.jpg"
  • />
  •  
  • <!--- Set the mime-type. --->
  • <cfcontent type="image/jpeg" />
  •  
  • <!---
  • Exit the request - we don't really need to do this --
  • this is here only to emphasize that ColdFusion is no
  • longer doing anything in this request that is realted to
  • the file delivery.
  • --->
  • <cfexit />
  •  
  •  
  • <cfelse>
  •  
  •  
  • <!---
  • The user has not purhcased this photo. Return an accessed
  • denied header response.
  • --->
  • <cfheader
  • statuscode="401"
  • statustext="Unauthorized"
  • />
  •  
  • <h1>
  • Access Denied
  • </h1>
  •  
  • <p>
  • You are not authorized to access this page.
  • </p>
  •  
  •  
  • </cfif>

Before it does anything else, this ColdFusion page checks to see if the user has actually purchased the photo. Remember, that's really the reason we are here - to figure out how to create a secure-access layer in ColdFusion in which we can perform user-specific security checks. But, once the permissions have been evaluated, ColdFusion simply defines a new HTTP header, "x-sendfile." The value stored in this header is the absolute file path of the secure file that we are streaming. Once ColdFusion finishes the request, the XSendFile Apache module then examines the outgoing HTTP headers and streams the given file back to the client.

NOTE: The X-SendFile header never reaches the client. Once the XSendFile module evaluates the outgoing HTTP headers, it strips the X-SendFile entry and streams the file at the defined path.

I have to say, MOD XSendFile is pretty darn awesome. Admittedly, I like to keep my application as modular as possible; that is, I like to keep as much as the application configuration inside the application itself. But, when you see the kind of power and functionality that can be provided by things like XSendFile and URL Rewriting, it definitely makes you a lot more open-minded. XSendFile makes it a breeze to provide ColdFusion-powered secure file access without putting an undue burden on the finite collection of ColdFusion threads.




Reader Comments

Apr 14, 2011 at 11:47 AM // reply »
2 Comments

This may be the first time I actually understood what you are doing on my first read-through. Really great example of integrating web server features with CF. Also, great example on how you integrate CF into Apache.


Apr 14, 2011 at 11:50 AM // reply »
10,743 Comments

@Brandon,

Ha ha, awesome - hopefully that means I am getting better at explaining myself :) I think this stuff is really cool. I can definitely see a lot of opportunities to use this module. I've never been truly happy with secure file downloads in the past.


Apr 14, 2011 at 11:58 AM // reply »
7 Comments

Thanks, Ben! I could have used this at my previous company, but now have this page bookmarked and indexed in Evernote so that I can figure out how to do it for my startup, as I know that file streaming will be needed.

I found your explanation easy to follow - you have always done a great job and continue to improve - we thank you!


Apr 14, 2011 at 12:17 PM // reply »
10,743 Comments

@Paul,

Thanks a lot! This has been a pretty feel-good day so far :D


Apr 14, 2011 at 2:17 PM // reply »
10 Comments

Been doing this and similar for the past 4 or 5 years. :) FYI, writing an IIS module for IIS7+ that does this is not very difficult (c#).

Prior to IIS7 it's a... hassle (ISAPI filters and native code), so I've never bothered.


Apr 14, 2011 at 3:54 PM // reply »
10,743 Comments

@Mark,

I haven't touched C# in a long time :) But, it's good to know that it's not a dead end in IIS. Glad this kind of approach has been proven over the past few years. I really like it!


Apr 15, 2011 at 7:05 AM // reply »
6 Comments

Interesting technique. Only problem is that it requires apache. I am using nginx as my front end web server. A little googling turned up an nginx module http://wiki.nginx.org/XSendfile that uses a very similar concept... so the same idea may work for me.

Thanks for the idea!


Apr 15, 2011 at 7:11 AM // reply »
26 Comments

Great idea!

We used the combination of mod_auth_cookie and Apache authentication to manage secure downloads, but your approach seems much better. Especially since mod_auth_cookie seems dead.

Thanks for pointing XSendfile.


Nov 22, 2011 at 1:08 PM // reply »
1 Comments

My apologies if my question is a bit basic, but is it possible to have Apache and CF running in separate servers to get this to work?

BTW also wanted to thank you, Ben. Since I recently started working with CF, I must say that I always find great information in your blog!


Post A Comment

Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.

Please review the following issues:

Author Name:


Author Email:

Author Website:

Comment:

Supported HTML tags for formatting: <strong>bold</strong>   <em>italic</em>   <code>code</code>







  • Help Wanted - Find Your Next ColdFusion Job
InVision App - Prototyping Made Beautiful With Prototyping Tools Ben Nadel's Company - Epicenter Consulting Recent Blog Comments
May 21, 2012 at 1:58 AM
Updated: Converting A ColdFusion Query To CSV Using QueryToCSV()
Hi Ben, why do you need to have so many double quotes when adding the field and field name to the row data? ----------------------------------------- <cfset LOCAL.RowData[ LOCAL.ColumnIndex ] = ... read »
AXL
May 21, 2012 at 1:24 AM
URL Rewriting And ColdFusion's WriteToBrowser Image Functionality (CFFileServlet)
@Mounir, Open your lower case URL Rewrite rule and add the following condition. Condition input: {REQUEST_URI} Check if input string: Does Not Match the Pattern Pattern: ^/CFFileServlet/_cf_ca ... read »
May 20, 2012 at 4:28 AM
Understanding The Complex And Circular Relationships Between Objects In JavaScript
@Will Vaughn I tried your javascript example but got this error:- foo.print is not a function ... read »
May 19, 2012 at 5:37 AM
A Graphical Explanation Of Javascript Closures In A jQuery Context
Thanks for this article, but I fear you missed an important point. If variables in the outer context change, these changes affect the inner anonymous functions as well. That means: if you change the ... read »
May 18, 2012 at 3:39 PM
Parsing CSV Data With An Input Stream And A Finite State Machine
Can you use file upload button with this? and read live? or does the file have to already be on the server saved? ... read »
May 18, 2012 at 1:06 AM
VIRGO (Aug. 23-Sept. 22): Dead On The Money!
A friend of mine and I were arguing about astrology and she told me that he believes in astrology. She hasn't provided me with any evidence that the belief makes any sense to me. She she been telling ... read »
May 17, 2012 at 11:32 PM
Using ColdFusion to Handle 404 Errors (Page Not Found) On Development Server
Very easy the configuration. I read a lot pages and I can't find the solution. I open the administrator and change this Administrator/server settings/Error Handlers/Missing Template Handler and p ... read »
May 17, 2012 at 3:13 PM
LOCAL Variables Scope Conflicts With ColdFusion Query of Queries
I never cease to be amazed that almost EVERY random CF issue I come across lands me on your site. Thank you for documenting your findings for the world. ... read »