ColdFusion ListGetAt() vs. GetToken()

Posted June 19, 2007 at 5:21 PM

Tags: ColdFusion

The other day I was asked which was better to use, ColdFusion's ListGetAt() or GetToken(). I think in 8 years of ColdFusion development I have used GetToken() only once or twice? Off hand, I didn't even know what it did exactly or specifically, what it did that was so different. From a cursory glance at the documentation, the two appears to be the same thing: they both take a list, an index, and a set of delimiters. It's not till I really looked at the ColdFusion documentation that I really saw that they are performing two similar but different functions.

Both functions do indeed get specific elements of a list, but they are different in two very important respects:

  1. LetGetAt()'s default delimiter is the comma. GetToken()'s default delimiter is actually a set of delimiters including the majority of white space characters (space, tab, and new line).
  2. If you reference an index that is beyond the end of the list, ListGetAt() will throw an exception where as GetToken() will just return an empty string.

To see this in action, I have set up a little demo in which we loop over the items in a list, going beyond the boundaries on purpose. For the first demo, we are going to ask both methods to use the comma as the list delimiter:

 Launch code in new window » Download code as text file »

  • <!--- Create out ColdFusion list. --->
  • <cfset strList = "Face,It,Maria,Bello,Is,Crazy,Hot!" />
  •  
  •  
  • <!---
  • To test how each method works, not only are we going
  • to loop over the elements in the list, we are going
  • to do so in an out-of-bounds fashion; notice that we
  • are starting before 1 and going until after the end
  • of the list.
  • --->
  • <cfloop
  • index="intI"
  • from="0"
  • to="#(ListLen( strList, ',' ) + 1)#"
  • step="1">
  •  
  • <!---
  • Because these methods might error, we are going
  • to wrap each one its own CFTry / CFCatch block
  • and display any caught errors.
  • --->
  • <p>
  • ListGetAt( #intI# ):
  •  
  • <cftry>
  • #ListGetAt( strList, intI, "," )#
  •  
  • <!--- Catch error and display message. --->
  • <cfcatch>
  • ERROR: #CFCATCH.Message#
  • </cfcatch>
  • </cftry>
  •  
  • <br />
  •  
  • GetToken( #intI# ):
  •  
  • <cftry>
  • #GetToken( strList, intI, "," )#
  •  
  • <!--- Catch error and display message. --->
  • <cfcatch>
  • ERROR: #CFCATCH.Message#
  • </cfcatch>
  • </cftry>
  • </p>
  •  
  • </cfloop>

Running the above code, we get the following output:

ListGetAt( 0 ): ERROR: Invalid list index 0.
GetToken( 0 ): ERROR: Parameter 2 of function GetToken which is now 0 must be a positive integer

ListGetAt( 1 ): Face
GetToken( 1 ): Face

ListGetAt( 2 ): It
GetToken( 2 ): It

ListGetAt( 3 ): Maria
GetToken( 3 ): Maria

ListGetAt( 4 ): Bello
GetToken( 4 ): Bello

ListGetAt( 5 ): Is
GetToken( 5 ): Is

ListGetAt( 6 ): Crazy
GetToken( 6 ): Crazy

ListGetAt( 7 ): Hot!
GetToken( 7 ): Hot!

ListGetAt( 8 ): ERROR: Invalid list index 8.
GetToken( 8 ):

Notice that when we go lower than the list length (index zero), both ColdFusion functions throw an error. While we are in the meat of the list, both function in exactly the same (with the explicit comma delimiter). Once we go past the end of the list, ListGetAt() throws an exception while GetToken() just returns an empty string.

Ok, but that's with both methods using explicit delimiters. Let's take a look at how GetToken() works if you let it use its default delimiter set. This time, our list will have line breaks and other white space rather than commas. For the ListGetAt(), we are going to build up a set of delimiters to use so that it finds more than just the first "element":

 Launch code in new window » Download code as text file »

  • <!---
  • Create a new list. This time, though, we are not going
  • to be using the comma as a delimiter; we are basically
  • going to be using whitespace as the delimiter.
  • --->
  • <cfsavecontent variable="strList">
  • Frances Mcdormand sort
  • of has it going on?!?
  • </cfsavecontent>
  •  
  • <!--- Set up the string of delimiters (white space). --->
  • <cfset strDelimiters = (
  • " " &
  • Chr( 13 ) &
  • Chr( 10 ) &
  • Chr( 9 )
  • ) />
  •  
  •  
  • <!---
  • To test how each method works, not only are we going
  • to loop over the elements in the list, we are going
  • to do so in an out-of-bounds fashion; notice that we
  • are starting before 1 and going until after the end
  • of the list.
  • --->
  • <cfloop
  • index="intI"
  • from="0"
  • to="#(ListLen( strList, strDelimiters ) + 1)#"
  • step="1">
  •  
  • <!---
  • Because these methods might error, we are going
  • to wrap each one its own CFTry / CFCatch block
  • and display any caught errors.
  • --->
  • <p>
  • ListGetAt( #intI# ):
  •  
  • <cftry>
  • #ListGetAt( strList, intI, strDelimiters )#
  •  
  • <!--- Catch error and display message. --->
  • <cfcatch>
  • ERROR: #CFCATCH.Message#
  • </cfcatch>
  • </cftry>
  •  
  • <br />
  •  
  • GetToken( #intI# ):
  •  
  • <!---
  • This time, notice that we are not passing
  • in any delimiters to the GetToken() method.
  • This will force GetToken() to use its default
  • set of delimiter characters.
  • --->
  • <cftry>
  • #GetToken( strList, intI )#
  •  
  • <!--- Catch error and display message. --->
  • <cfcatch>
  • ERROR: #CFCATCH.Message#
  • </cfcatch>
  • </cftry>
  • </p>
  •  
  • </cfloop>

Since ListGetAt() uses the comma by default in lists, we need to tell it to use all those white space characters as its list delimiters. Running the above code, we get the following output:

ListGetAt( 0 ): ERROR: Invalid list index 0.
GetToken( 0 ): ERROR: Parameter 2 of function GetToken which is now 0 must be a positive integer

ListGetAt( 1 ): Frances
GetToken( 1 ): Frances

ListGetAt( 2 ): Mcdormand
GetToken( 2 ): Mcdormand

ListGetAt( 3 ): sort
GetToken( 3 ): sort

ListGetAt( 4 ): of
GetToken( 4 ): of

ListGetAt( 5 ): has
GetToken( 5 ): has

ListGetAt( 6 ): it
GetToken( 6 ): it

ListGetAt( 7 ): going
GetToken( 7 ): going

ListGetAt( 8 ): on?!?
GetToken( 8 ): on?!?

ListGetAt( 9 ): ERROR: Invalid list index 9.
GetToken( 9 ):

Going outside the list boundaries works just like it did in the first demo; the real difference here is that GetToken(), without being given any explicit delimiters, was able parse the white-space-delimited list.

Ok, so nothing really revolutionary here, more just personal exploration; see, this is what happens when you don't read documentation - you don't even know what functions are supposed to do. At least now I know why these two functions coexist. I am not going to bother doing any speed tests on the two different methods because I don't use non-comma-delimited lists all that often, and so, I am not often in a position where GetToken() adds a functional benefit. I assume the two work in a similar fashion.

Download Code Snippet ZIP File

Comments (12)  |  Post Comment  |  Ask Ben  |  Permalink  |  Other Searches  |  Print Page



Adobe ColdFusion 8.0.1 Update - Helping Programmers To Be Signifanctly Less Girlie - Download ColdFusion 8 Update 8.0.1 Now.

Reader Comments

Ben, I've used getToken() when parsing input, importing a text file for example, because it allows compound delimiters. I've solved some sticky problems by taking advantage of that one feature.

Posted by Matthew Scott on Jun 19, 2007 at 6:03 PM


Good to hear it. The more tools we have, the better our solutions tend to be.

Posted by Ben Nadel on Jun 20, 2007 at 7:27 AM


Yeah, GetToken's ability to use multi-character delimiters is the only reason I would use it.

Posted by Steve on Jun 20, 2007 at 9:40 AM


Also realize, however, that ListGetAt() can also use multiple delimiters at the same time (which is what I am doing in the second example). But, explicit delimiters aside, GetToken() will not error if you go past the list, ListGetAt() will.

I think it comes down to a matter of convenience; if you are parsing something that uses white space as a delimiter, you should probably use GetToken() rather than using ListGetAt() with explicit delimiters.

Posted by Ben Nadel on Jun 20, 2007 at 9:49 AM


ListGetAt can use multiple delimiters, but each of them can be no more than one character in length. E.g., with ListGetAt, the delimiter value "test" is four discrete delimiters ("t", "e", "s", "t"), while with GetToken it is a single, four-character delimiter. IMO, that is a much more significant difference than the functions' default delimiters or whether they error outside of bounds.

Posted by Steve on Jun 20, 2007 at 12:24 PM


@Steve,

Very interesting. I think I missed that aspect. That is really cool. I will look at this more closely.

Posted by Ben Nadel on Jun 20, 2007 at 12:26 PM


@Steve,

I just did an experiment with the string:

Page1:Para4:;Page2:Para5

... and did a GetToken() passing in ":;" as the grouped delimiter thinking that it would only break between Para4 and Page2, however it seems to be breaking on the single colons as well.

Am I doing something wrong?

Posted by Ben Nadel on Jun 20, 2007 at 3:00 PM


Ben, I'm not sure. In fact, I only learned about the GetToken function from your post here, but it struck me as significant because I've often wanted to use multi-character delimiters with the CF list functions.

Here's how the LiveDocs describe the delimiter argument for GetToken:

"[...] A delimited list of delimiters. Elements may consist of multiple characters. [...] Default list delimiter: comma character."

URL: http://livedocs.adobe.com/coldfusion/7/htmldocs/00000502.htm

If it indeed doesn't work as advertised, that's a shame.

Posted by Steve on Jun 20, 2007 at 5:08 PM


Yeah, the documentation is very confusing. I don't understand that whole default list delimiter as comma in the context that it would work as you say it would. I think the documentation must be incorrect... either that or I am just totally missing something here.

Posted by Ben Nadel on Jun 20, 2007 at 5:21 PM


The same thing is happening to me- its delimiting with each character and not the string as a whole. This is very confusing since it isn't following the documentation

Posted by William on Jul 11, 2007 at 11:54 AM


@William,

I am not sure if is or is Not following the documentation. The documentation is confusing to me on this matter.

Posted by Ben Nadel on Jul 11, 2007 at 11:56 AM


I stumbled across this post's comments when trying to research why getToken()'s multi-char delimiter "functionality" wasn't working.

The comments for this post aren't much help of course, except to confirm that other folks can't get it to work either.

I seem to remember this working just fine previously; I'm suspicious it's a pre-MX vs post-MX issue?

In any case this Adobe page:
http://www.adobe.com/support/coldfusion/releasenotes/mx/mx61_known_problems.html#documentation
states:
"The CFML Reference entry for the GetToken function incorrectly states that the comma is the default list delimiter. There is no default list delimiter for the GetToken function. "

Lovely. This is why ColdFusion really bugs me sometimes. If you look at the documentation, they have not fixed it online, causing it to populate into CF 7 and 8 docs, even though as of CF 6 it's been known to be incorrect. Having wasted half my morning working on this (because this is *exactly* the functionality I need and it sure seemed possible), I'm rather frustrated at them. :-/ Particularly because it's not the only time their docs have been completely off the mark or very confusing.

Sigh, didn't mean to vent. Looks like I'll be developing my own multi-char delimiter parsing function. :-) I don't mind doing it....I just wish Adobe hadn't led me on a wild-goose-chase with their invalid docs.

Posted by Darren Cook on Aug 21, 2008 at 3:16 PM


Post Comment  |  Ask Ben


Home   |   Web Log   |   ColdFusion   |   Projects   |   Resume   |   Job Form   |   Search   |   Contact
Epicenter Consulting - Custom Software Solutions for Business Evolution HostMySite.com - The Leader In ColdFusion Hosting