Compiling Several Linked Files Into One File
I wanted to try and come up with something that was a "best of both worlds" kind of solution. And so, I called once again upon my friend, the ColdFusion custom tag. I am not 100% satisfied with this solution, but I have built a small set of ColdFusion custom tags that allow you to list the files in the HEAD tag, as usual, but will compile the files and then write the HTML to link to the final file. Take a look at this example:
Notice that all the scripts were compiled down to a single, subsequent HTTP request.
While I like this, the down side to a technique like this is that it has to process the ColdFusion custom tags for every single page request; the nice thing about a build script, like an ANT script, is that it only has to be done once. But, like I said, I don't like "compilation gestures"; I find them to be not intuitive to your average programmer. So, to get around this, I tried to make the ColdFusion custom tags as clean as possible and have them build the file only when necessary, taking super care to read from the file system as little as possible!
The best I could do was limit the file reads to a single FileExists() on the target file for each page request. I am not sure if I am satisfied with this. I really don't like the idea of reading from the file system for every single request - this could become a huge overhead if I am not careful. I think I might dump the idea of checking file existence and make the rebuild process something that can only be launched manually using the RebuildParam. After all, it's not like I have to rebuild the file for every request - once the file is built, my only real responsibly, which is the largest use-case, is to output the SCRIPT or LINK tags.
Anyway, here is what I have so far. This is the Files ColdFusion custom tag:
The File tag doesn't have much to it. It basically just collects file paths and stores them in the base tag (Files):
<!--- Check to see which mode we are executing. ---> <cfswitch expression="#THISTAG.ExecutionMode#"> <cfcase value="Start"> <!--- Get base tag data. ---> <cfset VARIABLES.FilesTag = GetBaseTagData( "cf_files" ) /> <!--- Param tag attributes. ---> <!--- This is the expanded path to the given file. ---> <cfparam name="ATTRIBUTES.Path" type="string" /> <!--- Check to see if we even need to worry about this file. Only go further if the base tags contains the rebuild flag. ---> <cfif NOT VARIABLES.FilesTag.Rebuild> <!--- We have no need to use the child tag, so exit it without processing. ---> <cfexit method="exittag" /> </cfif> <!--- ASSERT: At this point, we know that the compiled file needs to be rebuilt and therefore we will need to check this child tag. ---> <!--- Store path in base tag. Check to see if the file exists so that we only have valid files passed up to the base tag. ---> <cfif FileExists( ATTRIBUTES.Path )> <cfset ArrayAppend( VARIABLES.FilesTag.FilePaths, ATTRIBUTES.Path ) /> </cfif> </cfcase> <cfcase value="End"> </cfcase> </cfswitch>
So, there you have it. Like I said, I'm not fully satisfied with it, but I think I am headed in the right direction. I didn't include any kind of file compression in this version. When reading in the individual files, I could have done things like strip out white space and comments, but this was more a proof of concept than a final product.
Want to use code from this post? Check out the license.
One potential way around this would be to add a "build" number to the file name, perhaps from your source control, or the date of the last modified file from the set of combined files, or maybe use a GUID as part of the file name.
So thinking on this as I'm writing it, it may be worthwhile to generate a hash based upon all the file names that are being included into the page, and perhaps include the last modification dates of the files (maybe stored in an application scoped variable to prevent unnecessary disk access).
Anyway, just a couple of thoughts on what you're working through.
Brilliant idea! I like the hash. I think that takes care of all the issues that you mention. Thanks a lot.
if you have sessions enabled, you can just append session.sessionid to the url of the script:
I do this to all the script and link tags so that it only downloads once for the session instead of downloading on every page refresh.
Also a good suggestion, thanks.
Rather than doing it at runtime, why not do it at deploy time? We use a Ant script to aggregate all our JS/CSS files at build time, and then run the JS through the YUI Compressor. The result is nice modular JS/CSS at develop time and a compressed single file in production.
It may be a good idea to include some type of browser detection (cgi, perhaps) to append or substitute IE6 specific CSS.
How does that work with the linking in the HTML? The one thing I want to get around is having to explicitly link to a single file, assuming that it will be created with deployment. I think that is too "release oriented" and not aligned with the less than well planned world of web development.
One could still use conditional logic to include the File tags in this way. That is not something that I would want to move into the logic of the tag internals.
How about stripping out the line-breaks, tabs and whitespace?
It would save me the trip to the CSS/JS compressor. :)
As AJAX use picks up more and more, this type of utility becomes increasingly beneficial.
Here's how we do it, complete with code samples: http://www.barneyb.com/barneyblog/2008/04/08/build-time-aggregation-of-jscss-assets/
Figured that'd be easier than trying to communicate that in a comments discussion, though it probably robs Ben of some traffic. ;)
I'd hesitate to use "smart numbers" in your file name to force a browser to re-request. Instead, try using ETags. Shouldn't be difficult to use CF to insure that a new ETag is used.
What about using the new CF8 ajax goodness? Even doing a simple <cflayout> adds a ton of js and a few css files to your <head> section. There probably isn't any easy way to go back and grab those to put into one file.
You could repackage the aggregator as a Servlet filter, I suppose. Or fix CF's AJAX stuff. The former would probably be easier. ;)
Strangely enough, this is a problem I was getting ready to address in the next few days.
One of the things I wanted to tackle is loading of scripts outside of the main template. I use custom tags to drive all my UI components. I like my tags to load all the appropriate external files (style sheets, JS files etc.) That way I don't worry about missing dependencies.
Since I drive all my views through a central <cf_layout/> tag, I was going to make my link tag be usable from any file and then accumulate all the link files when my <cf_layout /> tag process the "end" mode.
As mentioned already, I was thinking of using a hash value of all the filenames combined together. The problem with concatentating all the files together per page request, is you could end up with several large files that are almost identical.
Of course, I may just end up deciding it's easier to place all my dependent scripts that I want to "merge" into a build directory and then just rebuild the scripts using an ANT script. That may end up being the most efficient method.
While I don't quite understand ETags or how they would be set, since the actual linked file is a JS/CSS file, I am not sure that ColdFusion would even get a hand in the matter. Do you have any more information / links on where CF and ETags have played together (just for my own curiosity).
I used to hate it that you couldn't flush content inside of a custom tag, but as I have gotten more and more into ColdFusion custom tags, I have grown to really love the fact that you have crazy control over the generated content up until the last close tag is executed.
That aside, I wouldn't be too worried about having large JS files being almost identical. After all, it's not like those similar files are being downloaded by the same client, right? Really, it just comes down to have some more files on the server, not a real concern these days. Yes, there is something to be said about things being clean as possible, but I wouldn't worry about it.