Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
I am the chief technical officer at InVision App, Inc - a prototyping and collaboration platform for designers, built by designers. I also rock out in JavaScript and ColdFusion 24x7.
Meanwhile on Twitter
Loading latest tweet...
Ben Nadel at RIA Unleashed (Nov. 2009) with:

Experimenting With List Comprehensions In ColdFusion

By Ben Nadel on
Tags: ColdFusion

I recently finished reading Seven Languages in Seven Weeks by Bruce Tate. The book gave a wonderful overview of seven different languages; and, while each language was rather different, they all seemed to have one thing in common: very powerful list manipulation features. A little while ago, I explored the concept of FoldLeft() in ColdFusion; today, I wanted to take a look at list comprehensions.

A list comprehension is basically a short-hand way of creating lists from existing lists. A list comprehension typically allows you to create the product of multiple lists and supply guard statements to make sure that each list item meets certain criteria. To see what I'm talking about, take a look at the following demo:

  • <!--- Create a list of girls. --->
  • <cfset girls = [ "Sarah", "Tricia", "Kit", "Joanna" ] />
  •  
  • <!--- Create a list of equations. --->
  • <cfset equations = [ "is", "is not" ] />
  •  
  • <!--- Create a list of adjectives. --->
  • <cfset adjectives = [ "hot", "awesome", "sexy" ] />
  •  
  • <!--- Merge the two collections. --->
  • <cfset newCollection = comprehension( "'##name## ##equation## so ##adjective##!' | name <- girls ; equation <- equations ; adjective <- adjectives ; (name neq 'Kit')" ) />
  •  
  • <!--- Output the new collection. --->
  • <cfdump var="#newCollection#" />

In this case, we have three lists: names, equations, and adjectives. Then, we are using a list comprehension to create a new list that contains every possible combination of the three lists according to the item statement and constrained by the guard statement.

When I run the above code, I get the following output:

1. Sarah is so hot!
2. Sarah is so awesome!
3. Sarah is so sexy!
4. Sarah is not so hot!
5. Sarah is not so awesome!
6. Sarah is not so sexy!
7. Tricia is so hot!
8. Tricia is so awesome!
9. Tricia is so sexy!
10. Tricia is not so hot!
11. Tricia is not so awesome!
12. Tricia is not so sexy!
13. Joanna is so hot!
14. Joanna is so awesome!
15. Joanna is so sexy!
16. Joanna is not so hot!
17. Joanna is not so awesome!
18. Joanna is not so sexy!

In general, a list comprehension follows this kind of structure:

item | list [, list ] [, guard]*

In ColdFusion, getting this to work was no easy task; and, the solution that I am presenting is rather incomplete. It was more of an experiment than a demonstration of how to get this done.

  • <cffunction
  • name="comprehension"
  • access="public"
  • returntype="array"
  • output="false"
  • hint="I perform a list comprehension on the input in the form of (item | list [; guard]) and return the resultant array.">
  •  
  • <!--- Define arguments. --->
  • <cfargument
  • name="specification"
  • type="string"
  • required="true"
  • hint="I am the comprehension specification."
  • />
  •  
  • <!--- Define the local scope. --->
  • <cfset var local = {} />
  •  
  • <!---
  • Create an array to hold the comprehension results. No
  • matter what kind of lists we are passed, an array will
  • always be returned.
  • --->
  • <cfset local.results = [] />
  •  
  • <!---
  • Get the item specification. This is the part of the
  • comprehension that defines the format and value of
  • the resultant collection item.
  • --->
  • <cfset local.itemSpecification = trim(
  • listFirst( arguments.specification, "|" )
  • ) />
  •  
  • <!---
  • Get the merge specification. This is the part of the
  • comprehension that tells us how to use the incoming
  • collections and what values to guard against.
  • --->
  • <cfset local.mergeSpecification = trim(
  • listRest( arguments.specification, "|" )
  • ) />
  •  
  • <!---
  • Define a collection for the collection statements used
  • in the comprehension.
  • --->
  • <cfset local.collections = [] />
  •  
  • <!---
  • Define a collection for the guard statements used in
  • the comprehension.
  • --->
  • <cfset local.guards = [] />
  •  
  • <!---
  • Define a variable to keep track of the product length.
  • This will be the size of the product of the max length of
  • each collection.
  • --->
  • <cfset local.productLength = 1 />
  •  
  • <!---
  • Loop over the merge specification to break the statements
  • into collection and guard statements.
  • --->
  • <cfloop
  • index="local.item"
  • array="#listToArray( local.mergeSpecification, ';' )#">
  •  
  • <!---
  • Check to see if the statement has an assignment operator
  • (left-arrow). If it does, then it is a collection
  • assignment; if it doesn't, then it is a guard statement.
  • --->
  • <cfif find( "<-", local.item )>
  •  
  • <!---
  • Since we know this is a collection statement, we want
  • to parse it now and normalize the collection type.
  • Ultimately, we want everything to be an array so that
  • it is easy to work with.
  • --->
  • <cfset local.collectionItem = {} />
  •  
  • <!--- Parse the name. --->
  • <cfset local.collectionItem.name = trim(
  • listFirst( local.item, "<" )
  • ) />
  •  
  • <!---
  • Parse the actual collection value - this will give us
  • a string which we can then evaluate to get the true
  • collection reference.
  • --->
  • <cfset local.collectionItem.collection = evaluate(
  • trim(
  • listRest( local.item, "-" )
  • )
  • ) />
  •  
  • <!---
  • Get the current index and max index. This will help
  • us when we need to loop over the collections.
  • --->
  • <cfset local.collectionItem.index = 1 />
  • <cfset local.collectionItem.maxIndex = 0 />
  •  
  • <!---
  • Check to see what type of a collection we are dealing
  • with. We want everything to be an array.
  • --->
  • <cfif isArray( local.collectionItem.collection )>
  •  
  • <!---
  • Nothing to do here - we just wanted to make sure
  • that this would match if it was true.
  • --->
  •  
  • <cfelseif isSimpleValue( local.collectionItem.collection )>
  •  
  • <!---
  • Assuming this is a list, convert it to an array.
  • Right now, we only support the comma as a
  • delimiter.
  • --->
  • <cfset local.collectionItem.collection = listToArray(
  • local.collectionItem.collection
  • ) />
  •  
  • <cfelse>
  •  
  • <!---
  • We could not determine the type of collection
  • that was passed-in.
  • --->
  • <cfthrow
  • type="InvalidCollection"
  • message="We could not determine the type of incoming collection in this comprehension."
  • detail="We could not determine the type of collection used in the merge statement, [#local.item#]."
  • />
  •  
  • </cfif>
  •  
  • <!---
  • Now that we have normalized the collection types,
  • let's update the maxIndex value to reflect the
  • length of the array.
  • --->
  • <cfset local.collectionItem.maxIndex = arrayLen(
  • local.collectionItem.collection
  • ) />
  •  
  • <!--- Update the product length. --->
  • <cfset local.productLength *= local.collectionItem.maxIndex />
  •  
  • <!--- Collection item. --->
  • <cfset arrayAppend(
  • local.collections,
  • local.collectionItem
  • ) />
  •  
  • <cfelse>
  •  
  • <!--- Guard item. --->
  • <cfset arrayAppend(
  • local.guards,
  • local.item
  • ) />
  •  
  • </cfif>
  •  
  • </cfloop>
  •  
  •  
  • <!---
  • Loop over the product length. At each index, we can use
  • the embedded collection index to figure out which value
  • to be using.
  • --->
  • <cfloop
  • index="local.productIndex"
  • from="1"
  • to="#local.productLength#"
  • step="1">
  •  
  •  
  • <!---
  • Loop over the collection to set the iteration value
  • for the collection. We need to use evaluate() here
  • because I thought it would too complex to try and
  • locally scope the assignments in conjunction with the
  • resultant item and the guard statements.
  • --->
  • <cfloop
  • index="local.collection"
  • array="#local.collections#">
  •  
  • <!--- Get the collection value. --->
  • <cfset evaluate( "#local.collection.name# = local.collection.collection[ #local.collection.index# ]" ) />
  •  
  • </cfloop>
  •  
  •  
  • <!---
  • Now that we have set the values, we need to figure out if
  • we need to use these values in the resultant item. Set a
  • flag that defaults to True; then, we can loop over the
  • guard statements.
  • --->
  • <cfset local.addResult = true />
  •  
  • <!--- Loop over the guard statements. --->
  • <cfloop
  • index="local.guardStatement"
  • array="#local.guards#">
  •  
  • <!--- Evaluate the guard statement. --->
  • <cfif !evaluate( local.guardStatement )>
  •  
  • <!---
  • This failed the guard statement, so don't create
  • a resultant item.
  • --->
  • <cfset local.addResult = false />
  •  
  • </cfif>
  •  
  • </cfloop>
  •  
  •  
  • <!---
  • Check to see if the guard statements all passed and we
  • can add a value.
  • --->
  • <cfif local.addResult>
  •  
  • <!---
  • Create the intermediary result item. Again, we need
  • to use evaluate here since none of the comprehension
  • statements are locally scoped.
  •  
  • NOTE: Evaluate() has some serious limitations with
  • implicitly created structs / arrays. Right now, we
  • can really only create simple values.
  • --->
  • <cfset local.resultItem = evaluate( local.itemSpecification ) />
  •  
  • <!--- Add the result. --->
  • <cfset arrayAppend(
  • local.results,
  • local.resultItem
  • ) />
  •  
  • </cfif>
  •  
  •  
  • <!---
  • Now that we've performed one iteration, increment the
  • index of each collection. We are going to do this
  • backwards since we are doing a depth-first merging.
  • --->
  •  
  • <!---
  • Set an increment value for the last collection - the last
  • collection will always increment.
  • --->
  • <cfset local.nextIncrement = 1 />
  •  
  • <cfloop
  • index="local.collectionIndex"
  • from="#arrayLen( local.collections )#"
  • to="1"
  • step="-1">
  •  
  • <!---
  • Increment the index of the current collection by the
  • defined increment.
  • --->
  • <cfset local.collections[ local.collectionIndex ].index += local.nextIncrement />
  •  
  • <!---
  • Reset the next increment - unless this collection is
  • maxed out, the previous one won't have to increment.
  • --->
  • <cfset local.nextIncrement = 0 />
  •  
  • <!---
  • Check to seee if the index is now greater than the
  • max index. If so, then reset it.
  • --->
  • <cfif (local.collections[ local.collectionIndex ].index gt local.collections[ local.collectionIndex ].maxIndex)>
  •  
  • <!--- Reset the current collection. --->
  • <cfset local.collections[ local.collectionIndex ].index = 1 />
  •  
  • <!--- Since we are resetting this collection, it means we need to increment the previous version. --->
  • <cfset local.nextIncrement = 1 />
  •  
  • </cfif>
  •  
  • </cfloop>
  •  
  •  
  • </cfloop>
  •  
  •  
  • <!--- Return the result. --->
  • <cfreturn local.results />
  • </cffunction>

As you can see in the above code, this solution relies heavily on the evaluate() statement in order to execute the various aspects of the list comprehension. Since the values in the list assignment can be used in both the guard statements and the item statement, I didn't want to even try to locally-scope them; for a proof-of-concept, that would have been far too much of a headache.

Unfortunately, evaluate() does have some serious limitations; the biggest of which is the fact that you can't put implicitly created arrays or structs in evaluated statements. As such, you can't use this list comprehension approach to create collections of complex values.

As I said in my review of Seven Languages in Seven Weeks, the book left with me one over-arching thought: list manipulation is super powerful. I'd like to see how I can get some more of that kind of functionality working in ColdFusion.




Reader Comments

@Andy,

I don't know Python itself; but, when I look at the solutions people have listed in my previous Seven Languages post, I can see that many languages just have so much awesome list-based functionality.

Reply to this Comment

List comprehension is powerful, but the downside is that when you deal with huge sets you are storing too much in RAM. So Python also has generator expressions which uses fancy tricks to only render an element when it is being iterated over. Which means you get huge speed boosts and better memory usage.

Plus, they are trivial to use:

  • # list comprehension in
  • big_list = [x for x in range(100000000000000)]
  •  
  • # generator expression
  • big_expression = (x for x in range(100000000000000))

If you rendered the big_list to screen you would see a lot of numbers. If you rendered big_expression, you would see something like <generator object <genexpr> at 0x1004ce5f0>,

All pretty nifty!

Reply to this Comment

@Daniel,

That's pretty cool. I did see in a number of the languages I looked at, they had this concept of "lazy sequences." It sounds like they worked in the same way - they weren't evaluated until they were used. So, you could even create infinite sequences and then just "take" a given number of items:

  • take 5 [1..]

This would take the first 5 elements form the infinite sequence starting at 1.

It's all a very interesting approach.

Reply to this Comment

Lazy sequences are a cornerstone of properly constructed ORMs. They actually only fire off SQL when they actually have to provide data back to the system. So for example using psuedo-ORM code:

  • records = Records.all()
  • records = records.filter(start_date > '2011/01/30')
  • records = records.filter(active=True)
  • print records

A good ORM will not do anything until that last print statement.

You can work a lot of finesse out of this. It is why Python's SQL Alchemy ORM is so awesome, because as the project has gotten older their lazy evaluation has gotten a lot more sophisticated. Django's ORM uses the same technique but not nearly as well.

I'm certain Hibernate does it. Or at least I hope so...

Reply to this Comment

@Daniel,

I believe you are right. I know that Hibernate, at the very least, batch-executes queries at the end of ORM sessions. I am pretty sure they also do lazy loading for composed objects (unless explicitly told not to). But, I don't have enough ORM experience to say anything for sure.

I read in Seven Languages book that a number of the languages do lazy evaluation of sequences, which is how you can have infinite sequences. Cool stuff!

Reply to this Comment

Post A Comment

You — Get Out Of My Dreams, Get Into My Comments
Live in the Now
Oops!
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.