Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
I am the chief technical officer at InVision App, Inc - a prototyping and collaboration platform for designers, built by designers. I also rock out in JavaScript and ColdFusion 24x7.
Meanwhile on Twitter
Loading latest tweet...
Ben Nadel at NCDevCon 2011 (Raleigh, NC) with:

Using CFThread Inside A ColdFusion Query Loop

By Ben Nadel on
Tags: ColdFusion

This morning, Sebastiaan and I have been having some great conversations about various ColdFusion features. When discussing CFThread, he asked whether or not query values should be explicitly passed into a CFThread tag (via its attributes), or if the CFThread tag can simply reference the query values implicitly. My guess was that due to asynchronous nature of the CFThread execution, one would have to bind the query values at thread-definition time (as attributes); but, I've never tried this myself so I wanted to give it a little test run.

 
 
 
 
 
 
 
 
 
 

In the following demo, I am manually building a ColdFUsion query and then looping over it twice. In the first loop, I am generating threads that refer to the query values as scoped to the primary page. In the second loop, I am again generating threads that refer to the query values; but this time, I am passing those values in as CFThread tag attributes.

  • <!--- Build a test query. --->
  • <cfset girls = queryNew( "" ) />
  •  
  • <!--- Add the name column. --->
  • <cfset queryAddColumn(
  • girls,
  • "name",
  • "cf_sql_varchar",
  • listToArray( "Tricia,Vicky,Erika" )
  • ) />
  •  
  • <!--- Add the description column. --->
  • <cfset queryAddColumn(
  • girls,
  • "description",
  • "cf_sql_varchar",
  • listToArray( "Athletic,Hot,Naughty" )
  • ) />
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!--- Loop over each query to launch a thread for each record. --->
  • <cfloop query="girls">
  •  
  • <!---
  • Launch a thread to test to see how the query records will
  • evaluate at the time the thread finally executes.
  • --->
  • <cfthread
  • name="thread#girls.currentRow#"
  • action="run">
  •  
  • <!---
  • Sleep the thread for a small amount of time to make
  • sure that following does not execute until after the
  • girls query has actually finished executing.
  • --->
  • <cfthread
  • action="sleep"
  • duration="500"
  • />
  •  
  • <cfoutput>
  • #girls.name# is #girls.description#
  • </cfoutput>
  •  
  • </cfthread>
  •  
  • </cfloop>
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!---
  • This time, rather than rely on variables-scoped query values,
  • we are going to explicitly pass the query into the CFThread tag
  • as tag attributes. This will perform a deep copy of the value
  • for the local scope.
  • --->
  •  
  • <!--- Loop over each query to launch a thread for each record. --->
  • <cfloop query="girls">
  •  
  • <!---
  • Launch a thread to test to see how the query records will
  • evaluate at the time the thread finally executes. Translate
  • the current query row values into CFThread tag attributes.
  • --->
  • <cfthread
  • name="thread_v2_#girls.currentRow#"
  • action="run"
  • girlname="#girls.name#"
  • girldescription="#girls.description#">
  •  
  • <!---
  • Sleep the thread for a small amount of time to make
  • sure that following does not execute until after the
  • girls query has actually finished executing.
  • --->
  • <cfthread
  • action="sleep"
  • duration="500"
  • />
  •  
  • <cfoutput>
  • #attributes.girlName# is #attributes.girldescription#
  • </cfoutput>
  •  
  • </cfthread>
  •  
  • </cfloop>
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!---
  • Join all the threads back so that we can examine the output
  • generated during their execution.
  • --->
  • <cfthread action="join" />
  •  
  • <!---
  • Output the threads (this should contain 6 threads - 3 from
  • each query iteration).
  • --->
  • <cfdump
  • var="#cfthread#"
  • label="CFThread Scope"
  • />

After both query iterations have passed, I am joining all of the threads back into the page and then CFDumping out the CFThread scope. This will give us access to the output generated by each thread (as defined by the available query values). When we run the above code, we get the following CFDump output:

 
 
 
 
 
 
Using CFThread Inside Of A ColdFusion Query Loop. 
 
 
 

Just as I suspected, you can see that the first three threads all have the same output. The reason that this happened is because the CFLoop had finished executing by the time the query values were referenced within the CFThread tag. As such, the column references pointed to whatever was available in the first row of the query (not what was available when the CFThread was defined). To get around this late-binding problem in the second query iteration, we pass the query column values into the CFThread tag as tag attributes. Since the CFThread tag is defined in the context of a given query record, the value binding is performed immediately and each CFThread now has a local copy of the appropriate query column value.

Passing data into and out of a CFThread tag is definitely a nuanced activity. Because the CFThread tag executes asynchronously to the main page and in no definite order in relation to other CFThread tags, we have to become much more conscious about the ways in which the data we are referencing might change unexpectedly before we have a chance use it.




Reader Comments

Thanks for the CFThread blog entries. I have a very difficult time wrapping my mind around the thread concept. So your entries have been very enlightening.

Reply to this Comment

@Don,

Happy to help. CFThread is at the same time awesome and tricky. I'm glad that some of this is starting to clear it up a bit.

Reply to this Comment

Ben,

This is so great ;-) So far away and still so close, working on similar stuff, me with a challenge, you with a solution or adaptation or information on the issue at hand. Thanx 4 the time you've put into this! My colleague and I are very greatful and hopefully we'll both of us see you in A'dam soon!

Sebastiaan

Reply to this Comment

@Sebastiaan,

My pleasure - thanks for the good conversational pieces. Looking forward to my EU tour :)

Reply to this Comment

@Ben - thanks for posting this. I recently started using CFTHREAD throughout my applications where certain actions could be processed in the background so the user doesn't have to sit there and wait for it to finish. I have been worried about not passing attributes to the tag directly and just what exactly was happening when it ran. Now I know :)

Reply to this Comment

@Grant,

Yeah, it can definitely be tricky, especially because the asynchronous nature of the execution can actually yield different results on different executions (if you are relying on variables that are not guaranteed).

Over time, you just start to get a better sense of what needs to be passed in and what doesn't. I am starting to see that bigger picture, but it took me a while to get here.

Reply to this Comment

@Ben,

Is there a way to CHECK to see if the query is finished instead of waiting .5 sec? Seems to me that if you enter 1,000,000 girls (you stud, you), then your query is going to take longer than a 500 msecs, and break this code.

Or are you saying that it's a moot point? You're tasking the thread to go do stuff, and only report back when it's ready for output?
-raj

Reply to this Comment

@Randall,

The 500 milliseconds is only to ensure that the logic of the CFThread might take longer than an individual iteration of the CFLoop. As long as it takes just a little bit longer - which may be due to processing power or simply due to thread queuing - then you'll run into the same problem.

Of course, I might also just not be fully understanding what's going on here. It is possible that CFThread never references the appropriate row of an "external" query... I am not 100% sure. All in all, you just want to make it a best practice to pass the values into the CFThread via the thread attributes.

Reply to this Comment

Ok, I think this is what I've been looking for but before I go through the process to test this in some code, I'd like to run my situation by you to see if it is the same concept that is described in this post.

I have a situation where I'm pretty sure cfthread is the right solution. I have a process I need to fire off (stored procedure and web service call) and due to several factors I can't guarantee this thing won't time out. So I wanted to nest these calls in a thread and essentially return the status of the thread to the user after something like 5 seconds. So I I have this:

<!-- start thread -->
<cfthread name="ABPWS-#form.xxxxxx#" action="run" priority="high">

<!-- first proceedure -->
<cfinvoke component="/gateway/mycfc" method="qryXxxxxxx" returnvariable="xxxxxx">
<cfinvokeargument name="xxxxxx" value="#application.xxxxxx#">
<cfinvokeargument name="xxxxxx" value="#application.xxxxxx#">
<cfinvokeargument name="xxxxxx" value="#xxxxxx#">
<cfinvokeargument name="xxxxxx" value="#xxxxxx#">
<cfinvokeargument name="xxxxxx" value="xxxxxx">
</cfinvoke>

<!-- second proceedure -->
<cfinvoke component="/gateway/mycfc" method="wsXxxxxxx" returnvariable="xxxxxx">
<cfinvokeargument name="xxxxxx" value="#application.xxxxxx#xxxxxx.asmx?wsdl">
<cfinvokeargument name="xxxxxx" value="#form.xxxxxx#">
<cfinvokeargument name="xxxxxx" value="#form.xxxxxx#">
</cfinvoke>

<cfthread action="join" name="ABPWS-#form.xxxxxx#" timeout="2" />
<!-- end thread -->

What I found was that anything inside this thread was nto getting executed and I think it is because of what you describe in this bog post. So, if, in my opening cfthread tag I weer to add all the necessary variables used in my cfinvokearguments, would that solve my problem? And then what would the dot notation be to output the thread status, evaluate(cfthread.ABPWS-#form.xxxxxx#.status)?

I hope this makes sense is appropriate to post here. I just can't seem to find any reasonable explanation for why processes in the thread are not being processed.

Thanks in advance.
(sorry if the post is really long - I broke the actual process down to the bare bones)

Reply to this Comment

@Derek,

When it comes to CFThread, if you're going to join the thread back to the page, then you're going to want to make sure that you have multiple threads trying to run in parallel. In your code snippet, I couldn't tell if the first and second procedure were both running in the same CFThread body, of they were were in different CFThread bodies.

If they are in the same CFThread body AND you are re-joining the thread, then the CFThread is not adding any value; in fact, it could delay the execution (since there's only so many threads the server can use at one time). If the two procedures are in two different threads, then you are hopefully getting some value.... assuming we can get them to run.

After you join the thread back to the page, dump out the CFThread scope:

  • <cfdump var="#cfthread#" />

This will give you some insight into what is going on with the thread. Threads don't just *not* work. Probably there is a bug that is causing an error that you're not seeing. By joining the thread and then dumping out the thread scope, you should be able to see the error in the thread object.

Try that and let us know. CFThread shares the same Variables scope of the parent page, so you don't always have to pass things in via the attributes (although a lot of the time is a good move... as long as you understand that it deep-copies every attribute).

Reply to this Comment

@Ben,

Thank you so much for the reply - the additional information you provided is very helpful. Every cfthread example I found online had code rejoining the thread, so it seemed like that was the common practice - I had no context (or didn't understand it). And, thinking back on this, there might have been data being passed into the thread that didn't meet the criteria for the process I was expecting to be completed by the thread - so at this point I can understand why I might not have been seeing the results I was expecting.

Ok, so to clarify a couple things that I think will put everything into perspective. The first and second procedure are in the same thread and the results of those end up being processed through a decision tree of other operations - all of which I have had the intention to be done in the background.

[20 minutes later]

Alright, I've got this figured out (thanks again). For the functionality I was looking for, I only need to remove the cfthread join statement in my code. I ended up finding the data issue that caused my outcome to be something other than what I was expecting. I cleaned up a couple other things as well to tie up loose ends and to document my logic.

So, THANK YOU!!! It's amusing to find myself, so often, just a hair away from "getting it" so I'm grateful to have fellow CFers, like yourself, with such a great online presence to turn to for help.

Have a great holiday.

Reply to this Comment

@Derek,

Awesome - glad you got it sorted out. CFThread can be a fun little journey to debug since it happens asynchronously. When in doubt, JOIN and CFDump :) That way, you can at least see what is going on in the parallel threads.

Happy holidays as well!

Reply to this Comment

A great Article, In fact, I found the solution of my problem. I am working on similar problem, feeling my neck struck into bottle and finding no other way to resolve it.

Now, I have found the root of problem by the courtesy of this article. Now, it is really a sign of relief I am feeling.

Thanks @Ben for your remarkable contributions

Reply to this Comment

Post A Comment

You — Get Out Of My Dreams, Get Into My Comments
Live in the Now
Oops!
Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.