Ionic ISAPI URL Rewrite Decodes "%26" Into Ampersand (&) By Default
Posted May 26, 2011 at 9:24 AM by Ben Nadel
Yesterday, I discovered a very confusing behavioral difference between a development server and a production server. Locally, query string variables could contain "%26" which represents a url-encoded ampersand. This escaped ampersand would then become decoded and available within the given URL variable at the ColdFusion level. On the production server, however, this "%26" was being decoded into a regular ampersand before the request was routed to the ColdFusion application server. As such, ColdFusion would treat the rendered "&" as a query string name-value-pair-delimiter and would end up creating an additional, invalid variable. After 3 hours of Googling, I finally figured out that this had to do with the default behavior of Ionic's ISAPI URL rewrite filter.
To illustrate what was happening, imagine that I was making a request to the ColdFusion server with the following query string:
?foo=this%26that
Locally, this would result in the URL name-value pair:
- foo = "this&that"
On the production server, however, we were getting the following URL name-value pairs:
- foo = "this"
- that = ""
As you can see, the premature decoding of "%26" into "&" caused ColdFusion to use it as a query string delimiter rather than as an embedded character.
Apparently, Ionic's ISAPI URL rewrite filter decodes certain values by default in order to make it easier for you to write your regular expression rules. This decoding gets applied to both the script name and the query string, which is what was causing the problem. To fix this situation, I had to add the following directive to the Ionic ISAPI config file:
- UrlDecoding OFF
This allowed the "%26" to fall through, unaltered, to the ColdFusion application server where it could be handled more appropriately.
This strikes me as a rather unusual default behavior for a URL rewriting engine. I have to assume that I am missing something critical about URL decoding as I can't think of a single reason why I'd want to handle decoding at the rewrite level vs. the underlying application level (whichever technology that may be).
Reader Comments
I had a similar problem a while ago. I was using an ajax post to send form variables but due to some funny interaction the server was creating the html simbol for an ampersand so my url varables were not sending through correctly.
my workaround was to send via form variable and instead of rewriting 1000's of lines of url's to forms I wrote a nested if statement to redo all the form variables as url variables. please note this process was faster than to rename all the url to form as the code was originally written badly.
@Herman,
I had considered looking at the POST angle as well; but, we did have one point of 3rd party contact where that wasn't a possibility (and was, in fact, where the problem first presented itself).
You can't imagine how frustrated I was getting trying to debug this problem!! :D
We've had other issues with Ionics & have had to roll back to an earlier version. Due to its recent verified inconsistency of returning the correct file, we've had to disable one of the rules that we were using (we added a 14 digit timestamp to the end of js/css resource filenames).
If you use 2.1.1.26 & SSL, make sure that "Site Logging" is enabled.
http://iirf.codeplex.com/workitem/27582
Could Adobe build a MODRewrite feature into ColdFusion or is there another, more dependable way of doing it in IIS?
BTW: I couldn't post this message using Firefox 4.0.1 (Windows7) and re-posted using Google Chrome. In FF Firebug, I see "$ is not defined". The local javascript file can load before the jQuery library since they are coming from 2 different domains & not loaded serially.
The versioning that you are using on resource files (?v=9) doesn't update some network caches properly & is the primary reason that I started adding a date hash to the end of filenames. Since you are serving up a CF file, I recommend adding it to the path instead and add the "js" extension to satisfy security software. (ie, javascript.cfm/v/9.js) Since using CF, make sure that you are returning "text/javascript" mime type (I see that you are doing this correctly.)
Here's the IIRF rule I am using:
RewriteCond %{SCRIPT_NAME} ^/_scripts/(.*)$ [I]
RewriteRule ^(.*)(_[0-9]{14}\.)(css|js)$ $1.$3 [I,QSA,L]
I maintain a server-scoped structure of the js/css filenames & dates. I wrote a function to access the structure and rewrite the resource URL & append the date/time hash if it exists. (Everything would be perfect if IIRF worked consistently... to many reports from clients that it wasn't working. sigh.)
@James,
Locally, I use Apache mod_rewrite, and on this blog, personally, I use IIS MOD-Rewrite. I only have Ionic running on one particular production site; I've not had occasion to test this anywhere else (as I very rarely have escaped ampersands in anything).
That's not good to hear about the "$" undefined issue! The two script files *should* load in serial, even if they are retreived in parallel. From my understanding, the browser will optimize the HTTP connections (in parallel); but, once gotten, it will still execute the Script tags in the order in which they were defined in the DOM.
I haven't made any changes to the JS in some time, so I don't think it would be a caching issue.
I'll see if I can figure it out - thanks for letting me know!
@Ben:
I wonder if it's really not a bug in Ionic. My guess is it's not really supposed to be passing back the decoded parameters. I can't think of a single situation in which that wouldn't ultimately cause potential issues.
@Dan,
Right, the only reason I can think of it is that was intended only for the "file path" part of the URL. Since certain characters are illegal in the URL, that would be the only place where it would make pattern matching easier.
Given the fact that using illegal characters in URLs it a bad idea to begin with, I certainly can't think of a reason it would make sense for the query string.
This happened to me today in fact. The problem was that Ionic was taking spaces in file names, decoding them, and then when IIS got it, it got "%20" literally-- not the encoded version--thus IIS couldn't decode it and I got a 404. At least, that's what appeared to be happening. The funny thing was that it only happened if I was trying to pass the URL into the CGI.Path_Info and not into a query.
I guess I'm lucky you addressed this issue just a few days ago, otherwise I'd probably still be pulling my hair out.
@Doug,
Awesome, I'm glad that I could help. Seems like such a frustrating problem to waste time on. That's weird though that it only affected the path_info variable. Very odd!
Does anyone use any other mod-rewriting tools for IIS?
I recently found "IIS Mod-Rewrite". It's not free, but it has 100% compatible syntax and behavior with Apache mod_rewrite and override (.htaccess) configurations.
http://www.micronovae.com/ModRewrite/
I'd like to know how it compares to IIRF (or other solutions). I've had to disable some IIRF rules since IIS didn't seem to be consistently returning the correct files.
I'm considering using this reverse proxy product for hosting resource files with a datehash added to the end of filenames (for permanent caching/cache busting). It handles rewriting rules and seems more robust than IIS (15,000 requests/sec):
http://www.iqproxyserver.com/
@James,
I actually use IIS Mod-Rewrite for this blog:
I've been very happy with it. It seems to be extremely compatible with my local Apache mod_rewrite module, which makes mirroring live and local a snap!
