Skip to main content
Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.

deserializeJson() Silently Fails On High-Precision Numbers In Lucee CFML 5.3.4.77

By Ben Nadel on
Tags: ColdFusion

Yesterday, Nicholas Mackey - from our Inspect team - stumbled upon an interesting behavior in which a JSON (JavaScript Object Notation) payload coming out of the Craft plug-in was including a Float value with something like 20-digits of precision. I have no idea how that value was being calculated on the Craft side; but, the interesting part of this is that when Lucee CFML 5.3.4.77 went to deserialize this JSON payload, it silently failed on said high-precision value, leaving it as a String, not a Number.

We weren't sure where in the data-processing pipeline the transformation was failing. So, I set up a small test for the first step in the pipeline: deserializing the JSON payload.

<cfscript>

	```
	<cfsavecontent variable="jsonPayload">
		{
			"a": 1.1,
			"b": 1.01,
			"c": 1.001,
			"d": 1.0001,
			"e": 1.00001,
			"f": 1.000001,
			"g": 1.0000001,
			"h": 1.00000001,
			"i": 1.000000001,
			"j": 1.0000000001,
			"k": 1.00000000001,
			"l": 1.000000000001,
			"m": 1.0000000000001,
			"n": 1.00000000000001,
			"o": 1.000000000000001,
			"p": 1.0000000000000001,
			"q": 1.00000000000000001,
			"r": 1.000000000000000001,
			"s": 1.0000000000000000001,
			"t": 1.00000000000000000001,
			"u": 1.000000000000000000001,
			"v": 1.0000000000000000000001,
			"w": 1.00000000000000000000001,
			"x": 1.000000000000000000000001,
			"y": 1.0000000000000000000000001,
			"z": 1.00000000000000000000000001
		}
	</cfsavecontent>
	```

	dump( deserializeJson( jsonPayload ) );

</cfscript>

Here, you can see that I have a stringified version of a Struct-literal (via CFSaveContent). The values in the struct are floats with increasing precision. And, when we run this ColdFusion code in Lucee CFML 5.3.4.77, we get the following output:

Output of deserializeJson() function shows mix of Numeric and String data-types in Lucee CFML.

As you can see, half the values get parsed into numbers; half the values get parsed into strings.

ASIDE: If you add more digits to the "whole number" portion of the value, Lucee will accept fewer decimal places. This has to do with the underlying Java data type that is being used, which can accurately represent only a certain amount of data. I believe this relates to a 32-bit value container; but, honestly, I don't really understand the low-level mechanics of data-types and precision.

It would be easy to call this a "bug", since numbers are getting parsed as strings. However, I do not believe this is a bug; because, what's the alternative? Unless Lucee were going to parse the value into a long, which is not the number-type is typically uses for numbers, the only other outcome would be throwing an error. And, if it threw an error, we wouldn't be able to see the data at all. At least with the current implementation, you get to see the data. And, from there, you can "transform it" programmatically to something that can fit into the Lucee CFML data-type scheme.

For example, we can take the stringified numbers and convert them into less-precise numbers. As Bryan Stanly pointed out, we can just round() the values to cast them to a lower-precision value:

<cfscript>

	```
	<cfsavecontent variable="jsonPayload">
		{
			"a": 1.1,
			"b": 1.01,
			"c": 1.001,
			"d": 1.0001,
			"e": 1.00001,
			"f": 1.000001,
			"g": 1.0000001,
			"h": 1.00000001,
			"i": 1.000000001,
			"j": 1.0000000001,
			"k": 1.00000000001,
			"l": 1.000000000001,
			"m": 1.0000000000001,
			"n": 1.00000000000001,
			"o": 1.000000000000001,
			"p": 1.0000000000000001,
			"q": 1.00000000000000001,
			"r": 1.000000000000000001,
			"s": 1.0000000000000000001,
			"t": 1.00000000000000000001,
			"u": 1.000000000000000000001,
			"v": 1.0000000000000000000001,
			"w": 1.00000000000000000000001,
			"x": 1.000000000000000000000001,
			"y": 1.0000000000000000000000001,
			"z": 1.00000000000000000000000001
		}
	</cfsavecontent>
	```

	// Since we know some of the numeric values may come back with a higher-precision
	// that Lucee can handle, we can cast all the values to numbers with lower-precision.
	rounededData = deserializeJson( jsonPayload ).map(
		( key, value ) => {

			return( round( value, 5 ) );

		}
	);

	dump( rounededData );

</cfscript>

As you can see, we're passing each value to round(), which in our case, will implicitly cast the stringified numbers to numbers. And, when we run this ColdFusion code, we get the following output:

Output of deserializeJson() function with mapped round() results in all numeric values in Lucee CFML.

As you can see, all of the data-types in this case are numeric. Of course, the stringified values with 20-digits of decimal precision are simply rounded to whole-numbers.

Unfortunately, since this data gets persisted in our system using MongoDB - which is schemaless - we probably have Documents that contain a mixture of value-types. So, in order to normalize the data for the client, we'll likely have to do the rounding on both the input and the output so that we can fix incoming data, but also gracefully handle existing data.



Reader Comments

What has two thumbs and hopes you leave a comment? This Guy! (Ben Nadel).

Post A Comment

You — Get Out Of My Dreams, Get Into My Blog
Live in the Now
Oops!
NEW: Some basic markdown formatting is now supported: bold, italic, blockquotes, lists, fenced code-blocks. Read more about markdown syntax »
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.