Comparing ColdFusion Number Randomization Algorithms
The other week, I posted about how I didn't feel like ColdFusion always did a great job of random number generation. When using ColdFusion's RandRange() method, I just pass in the two required integers. After posting this, Dustin told me that ColdFusion MX7 introduced a third argument for the RandRange() method, which was the algorithm by which the random numbers were generated. By default, ColdFusion uses the CFMX_COMPAT algorithm. Apparently (and this is stated directly in the documentation), the alternate algorithm, SHA1PRNG, will do a much better job of randomizing numbers.
To explore these two algorithms, I first wanted to start out graphing them, to see if I could see any obvious trends. In the following example, I am looping over the two algorithms and then using ColdFusion's CFChart tag to charge 50 random numbers between the values 1 and 50.
| <!--- | |
| Loop over the two algorithms, the default | |
| CFMX_COMPAT and then the SHA1PRNG. We are | |
| going to chart some random numbers to see | |
| what they look like. | |
| ---> | |
| <cfloop | |
| index="strAlgorithm" | |
| list="CFMX_COMPAT,SHA1PRNG" | |
| delimiters=","> | |
| <!--- | |
| Create a line graph of this randomly | |
| selected numbers. | |
| ---> | |
| <cfchart | |
| format="png" | |
| chartheight="500" | |
| chartwidth="545" | |
| labelformat="number" | |
| xaxistitle="Iteration" | |
| yaxistitle="Random Number"> | |
| <cfchartseries type="line"> | |
| <!--- | |
| Create each data item by randomly generating | |
| a number using one of the algorithms. | |
| ---> | |
| <cfloop | |
| index="intI" | |
| from="1" | |
| to="50" | |
| step="1"> | |
| <cfchartdata | |
| item="#intI#" | |
| value="#RandRange( 1, 50, strAlgorithm )#" | |
| /> | |
| </cfloop> | |
| </cfchartseries> | |
| </cfchart> | |
| </cfloop> |
From the above code, we get the following graphs:
Algorithm: CFMX_COMPAT (ColdFusion's Default)
Algorithm: SHA1PRNG (Added in ColdFusion MX7)
Now, I look at these two graphs, and frankly, they don't mean anything to me. I don't see trends, and even if I do see some trends, I don't understand the significance. Both of these graphics look like a nice randomized set of numbers.
But more than that, this is not a useful test case for me. My issues with randomness rarely involve generating a ton of numbers right in a row; my scenarios usually involve manually refreshing a page to see if something is rotating "properly" (think advertisements or header images). In that case, there is a big delay between random number generation (compared the delay between CFLoop iterations). In my next experiment, I am using a META tag refresh to put a uniform delay between my page refreshes as this will most closely mimic me sitting there and hitting the browser's refresh button:
| <!--- Param the list of random numbers. ---> | |
| <cfparam | |
| name="URL.numbers" | |
| type="string" | |
| default="" | |
| /> | |
| <!--- | |
| Create a random number using one of the | |
| two algorithms, CFMX_COMPAT or SHA1PRNG. | |
| ---> | |
| <cfset intNumber = RandRange( 1, 10, "SHA1PRNG" ) /> | |
| <!--- Add it to the list of numbers. ---> | |
| <cfset URL.numbers = ListAppend( URL.numbers, intNumber ) /> | |
| <!--- | |
| Check to see if we have generated enough numbers. | |
| We want to generate 20. If have less than 20, let | |
| provide the refresh link. If we have 20, just output | |
| the numbers. | |
| ---> | |
| <cfif (ListLen( URL.numbers ) LT 20)> | |
| <!--- | |
| Provide meta-drive refresh. This is to ensure | |
| that the timing of the refresh is similar for | |
| each page refresh. | |
| ---> | |
| <meta | |
| http-equiv="refresh" | |
| content=".5; url=#CGI.script_name#?numbers=#URL.numbers#" | |
| /> | |
| <cfelse> | |
| <!--- We have all the numbers, so output them. ---> | |
| #URL.numbers# | |
| </cfif> |
I ran the above code three times for each algorithm and here are the number lists that were generated:
Algorithm: CFMX_COMPAT (ColdFusion's Default)
9,7,7,7,7,7,7,6,6,6,6,9,9,8,9,8,8,8,8,8
3,6,6,6,5,6,6,5,5,4,4,5,2,3,2,2,2,2,2,2
5,8,8,7,7,6,7,7,6,6,9,10,9,9,8,8,9,8,8,8
Algorithm: SHA1PRNG (Added in ColdFusion MX7)
6,8,9,6,4,5,5,9,8,4,10,5,7,4,4,8,10,2,6,1
6,6,8,9,8,1,8,9,9,3,7,3,3,5,6,7,7,3,4,5
4,8,2,8,4,1,4,4,3,3,9,10,2,10,6,6,4,4,5,2
Just looking at these numbers, I can clearly see grouping in the CFMX_COMPAT algorithm. There is some grouping in the SHA1PRNG algorithm, but to a much much lesser degree. I don't know how the timing of the random number generation affects things, but it seems to have some sort of a link to the seemingly effective nature of the outcome. Now, I say "seemingly" because, remember, I am really more concerned about an even distribution of numbers and less so about the actual randomization of the numbers. Randomization or not, the SHA1PRNG seems to have a better distribution of numbers.
Want to use code from this post? Check out the license.
Reader Comments
Here's the mathematical proof for the variance (http://en.wikipedia.org/wiki/Variance) of the above results.
CFMX_COMPAT - variance - standard dev
test 1 - 1.05 - 1.0246
test 2 - 2.69 - 1.6401
test 3 - 1.5275 - 1.2359
SHA1PRNG - variance - standard dev
test 1 - 6.1475 - 2.4794
test 2 - 5.4275 - 2.3296
test 3 - 7.1475 - 2.6734
As the numbers get larger, the more variance the test set has present. A standard deviation of 2 means you should be hitting ~95% of your population, where 1 is only ~68%. The proof is in the pudding so to speak (sorry for the math pun).
I used this UDF to calculate the variance: http://www.cflib.org/udf.cfm?ID=256
@Dustin,
It's been a million years since I took a statistics class (and didn't do so well in it). From what it looks like, a bigger standard deviation is a Good thing since, if you think about the Bell Curve, you are covering more ground (as you say, I think). Thanks for doing the testing.
Yeah, it has been a long time for me as well. In fact I forgot about normal vs. random distributions when talking about standard deviation. The confidence levels stated (~95% and ~64%) are for a normal distribution, which we aren't dealing with. For a random distribution it is ~75% for 2 stddevs and ~50% for 1.41 stddevs.
I'm beginning to remember why I didn't particularity care for the class.
Yeah, if I never hear about a z-test or t-square test (or something like that) again, I will quite content. My brain just doesn't seem to like that sort of thing.
Thanks Ben,
I was complaining about the default behavior of randrange to a colleague only 2 days ago when observing the behavior of an online competition application we were running.
Little did I know that this behavior could be changed!
All goes to show I should RTFM!
@Dan,
Hey, I just learned about this too :)
I will quite content. My brain just doesn't seem to like that sort of thing.