Ben Nadel
On User Experience (UX) Design, JavaScript, ColdFusion, Node.js, Life, and Love.
I am the chief technical officer at InVision App, Inc - a prototyping and collaboration platform for designers, built by designers. I also rock out in JavaScript and ColdFusion 24x7.
Meanwhile on Twitter
Loading latest tweet...
Ben Nadel at cf.Objective() 2009 (Minneapolis, MN) with:

Printing To The Primary Output Stream From Within A ColdFusion CFThread Tag

By Ben Nadel on
Tags: ColdFusion

A lot of the time, when we use ColdFusion's CFThread tag, we use it in a "set it and forget it" style approach; that is, we spawn a parallel thread and don't wait for it to come back. Other times, however, we do wait for the parallel threads to finish processing so that we can examine their status or take advantage of the precipitated state change. In such cases - where we wait for the thread to finish - I though it would be interesting to get the thread to write output to the parent page.

 
 
 
 
 
 
 
 
 
 

All rendered output in ColdFusion is written to an output stream. Each page in ColdFusion has its own page context. Part of the definition of the page context is the output stream to which the generated content will be written. In some cases, this output stream is the one output stream that gets flushed to the client; in other cases, the output stream is a separate output stream that the client may never know about.

When it comes to ColdFusion's CFThread tag, we can quickly see that each CFThread tag has its own separate output stream. This is why content generated within a CFThread tag does not get written to the output stream that is flushed to the client. But, if the page context is an object that be accessed (via getPageContext()), can we use the root page context as a way to let our CFThread tags hook into the primary output stream?

To start this experiment, let's create a ColdFusion custom tag that writes its own generated content to the output stream of a given page context.

output.cfm (ColdFusion custom tag)

  • <!---
  • Check to see if we are running in the end mode of the tag.
  • That is the only mode where we will have a page context to
  • work with.
  • --->
  • <cfif (thisTag.executionMode eq "end")>
  •  
  • <!---
  • Param the page context. This is the context that contains
  • the output buffer to which we are writing.
  • --->
  • <cfparam
  • name="attributes.pageContext"
  • type="any"
  • />
  •  
  • <!--- Get the response object. --->
  • <cfset response = attributes.pageContext.getResponse() />
  •  
  • <!--- Get the character-based output stream. --->
  • <cfset outputStream = response.getOutputStream() />
  •  
  • <!---
  • Print the generated content of this custom tag to the
  • given output stream.
  • --->
  • <cfset outputStream.print(
  • javaCast( "string", thisTag.generatedContent )
  • ) />
  •  
  • <!---
  • Clear the generated content so that we aren't printing it
  • to two different output buffers.
  • --->
  • <cfset thisTag.generatedContent = "" />
  •  
  • </cfif>

As you can see, this ColdFusion custom tag takes one attribute - pageContext. This is going to be some instance of the object returned by a call to getPageContext(). Once the custom tag has finished executing, it explicitly accesses the output stream of the given page context and writes its generated content to that output stream.

Ok, now that we have a way to write content to the output stream associated with an arbitrary page context, let's try using this within a CFThread tag. In the following demo, I am going to spawn two parallel threads that each download an array of images. As each image is downloaded, the respective CFThread tags will provide user-feedback by writing status statements to the primary page's output stream.

  • <!---
  • Get a reference to the parent page context. This will be the
  • context to which our CFThread instances will print. Since each
  • CFThread shares the Variables scope with the spawning context,
  • this variable will be available without needed a deep-copy (ie.
  • without passing via the CFThread tag attributes).
  • --->
  • <cfset rootPageContext = getPageContext() />
  •  
  •  
  • <!---
  • Build an array of image URLs to download (we don't care about
  • the images, we just want something that will take a while to
  • process).
  • --->
  • <cfset imageUrls = [
  • "http://farm4.static.flickr.com/3191/2395355756_507b1940e4_b.jpg",
  • "http://farm3.static.flickr.com/2289/2395352122_65a666429f_b.jpg",
  • "http://farm4.static.flickr.com/3022/2394516701_bf4eccee3b_b.jpg",
  • "http://farm4.static.flickr.com/3227/2395350024_778bd01df7_b.jpg",
  • "http://farm3.static.flickr.com/2168/2394514421_25c4483573_b.jpg",
  • "http://farm3.static.flickr.com/2303/2395347746_97c2814944_b.jpg",
  • "http://farm4.static.flickr.com/3214/2395343244_7795c8de09_b.jpg",
  • "http://farm4.static.flickr.com/3022/2395368874_18fa5f189f_b.jpg",
  • "http://farm4.static.flickr.com/3017/2395367726_bd0c99f397_b.jpg",
  • "http://farm4.static.flickr.com/3280/2394532393_99a0545761_b.jpg",
  • "http://farm3.static.flickr.com/2179/2395365392_b1e9c5a45c_b.jpg",
  • "http://farm3.static.flickr.com/2117/2394527499_5b7e704b9c_b.jpg",
  • "http://farm4.static.flickr.com/3267/2395360732_7bc0266c0b_b.jpg",
  • "http://farm3.static.flickr.com/2241/2395359786_dc8006f0f8_b.jpg",
  • "http://farm3.static.flickr.com/2130/2394524263_a22f7e3732_b.jpg",
  • "http://farm4.static.flickr.com/3189/2395329372_718e759834_z.jpg"
  • ] />
  •  
  •  
  • <!--- Define the path to which we are going to save the images. --->
  • <cfset imagesDirectory = expandPath( "./images/" ) />
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!---
  • Now, let's launch a thread that downloads all of the images to
  • the local file system. After each image, the thread is going to
  • print to the root page output.
  • --->
  • <cfthread
  • name="threadA"
  • action="run">
  •  
  • <!--- Loop over the array to download each image. --->
  • <cfloop
  • index="index"
  • from="1"
  • to="#arrayLen( imageUrls )#"
  • step="1">
  •  
  • <!--- Download the image binary. --->
  • <cfhttp
  • result="imageGet"
  • method="get"
  • url="#imageUrls[ index ]#"
  • getasbinary="auto"
  • />
  •  
  • <!--- Save the image binary. --->
  • <cffile
  • action="write"
  • output="#imageGet.fileContent#"
  • file="#imagesDirectory#A_#index#.jpg"
  • />
  •  
  • <!---
  • Write the feedback to the root output. Notice that we
  • are passing in a reference to the page context of the
  • root page, NOT the CFThread.
  • --->
  • <cf_output pagecontext="#rootPageContext#">
  • <cfoutput>
  •  
  • Downloaded Thread A - #index#<br />
  •  
  • </cfoutput>
  • </cf_output>
  •  
  • </cfloop>
  •  
  • </cfthread>
  •  
  •  
  • <!---
  • Now, let's launch another thread that does the exact same thing
  • as the first thread. This is just to show the parallal nature of
  • multiple threads all writing to the same output.
  • --->
  • <cfthread
  • name="threadB"
  • action="run">
  •  
  • <!--- Loop over the array to download each image. --->
  • <cfloop
  • index="index"
  • from="1"
  • to="#arrayLen( imageUrls )#"
  • step="1">
  •  
  • <!--- Download the image binary. --->
  • <cfhttp
  • result="imageGet"
  • method="get"
  • url="#imageUrls[ index ]#"
  • getasbinary="auto"
  • />
  •  
  • <!--- Save the image binary. --->
  • <cffile
  • action="write"
  • output="#imageGet.fileContent#"
  • file="#imagesDirectory#B_#index#.jpg"
  • />
  •  
  • <!---
  • Write the feedback to the root output. Notice that we
  • are passing in a reference to the page context of the
  • root page, NOT the CFThread.
  • --->
  • <cf_output pagecontext="#rootPageContext#">
  • <cfoutput>
  •  
  • Downloaded Thread B - #index#<br />
  •  
  • </cfoutput>
  • </cf_output>
  •  
  • </cfloop>
  •  
  • </cfthread>
  •  
  •  
  • <!--- ----------------------------------------------------- --->
  • <!--- ----------------------------------------------------- --->
  •  
  •  
  • <!--- Flag that the threads are not done yet. --->
  • <cfset isThreadsProcessing = true />
  •  
  • <!---
  • Keep looping until the threads are done. We could use a JOIN to
  • wait for the parallel threads to finish; however, that would
  • pause the output of the parent thread and we would not get any
  • real-time feedback.
  • --->
  • <cfloop condition="isThreadsProcessing">
  •  
  •  
  • Downloading...<br />
  •  
  • <!---
  • Flush any output generated be either this page or one of
  • the threads.
  • --->
  • <cfflush interval="1" />
  •  
  • <!--- Check to see if the threads are done. --->
  • <cfset isThreadsProcessing = !(
  • listFindNoCase( "completed,terminated", cfthread.threadA.status ) &&
  • listFindNoCase( "completed,terminated", cfthread.threadB.status )
  • ) />
  •  
  • <!---
  • If the threads are still processing, then sleep the parent
  • page for a bit. That will give the parallel threads time to
  • make some progress and write to the parent page's output
  • buffer.
  • --->
  • <cfif isThreadsProcessing>
  •  
  • <!--- Sleep the parent page briefly. --->
  • <cfthread
  • action="sleep"
  • duration="500"
  • />
  •  
  • </cfif>
  •  
  •  
  • </cfloop>
  •  
  •  
  • Done downloading!<br />

After I have spawned my two parallel threads, you can see that I am waiting for both of them to finish. I could have used a CFThread/Join action to accomplish this; however, when you wait for threads to "join", the parent page appears to be put to sleep. At the very least, the parallel threads cannot flush the root page output stream while the root page is waiting for the join. By manually checking the CFThread status and then explicitly sleeping the root page, it gives me control over how the root page output stream is flushed to the client.

When we run the above code, we get the following output (which is better represented in the video):

Downloading...
Downloading...
Downloaded Thread B - 1
Downloading...
Downloaded Thread A - 1
Downloading...
Downloaded Thread A - 2
Downloaded Thread B - 2
Downloading...
Downloading...
Downloaded Thread A - 3
Downloaded Thread B - 3
Downloading...
Downloaded Thread A - 4
Downloaded Thread B - 4
Downloading...
Downloading...
Downloaded Thread A - 5
Downloaded Thread B - 5
Downloading...
Downloaded Thread B - 6
Downloading...
Downloaded Thread A - 6
Downloading...
Downloaded Thread B - 7
Downloaded Thread A - 7
Downloading...
Downloaded Thread B - 8
Downloading...
Downloaded Thread B - 9
Downloading...
Downloading...
Downloaded Thread A - 8
Downloaded Thread B - 10
Downloading...
Downloaded Thread A - 9
Downloading...
Downloaded Thread B - 11
Downloading...
Downloaded Thread A - 10
Downloaded Thread B - 12
Downloading...
Downloaded Thread A - 11
Downloading...
Downloaded Thread B - 13
Downloaded Thread A - 12
Downloading...
Downloaded Thread B - 14
Downloading...
Downloaded Thread B - 15
Downloaded Thread A - 13
Downloading...
Downloaded Thread B - 16
Downloading...
Downloaded Thread A - 14
Downloading...
Downloaded Thread A - 15
Downloaded Thread A - 16
Downloading...
Done downloading!

As you can see, each thread was able to print status statements to the root page's output stream so as to provide real-time feedback to the client.

I don't know how often, of even if I would ever use an approach like this; but, as an experiment, I just wanted to see if it could be done. I've looked deeply into bi-directional ColdFusion CFThread tag communication and this is just another way to get information out of a thread and into the parent page.




Reader Comments

I'm not sure if that OutputStream is thread-safe or not (and I can't find any confirmation via my Google-foo). You may need some locking or syncrhonization around the print() calls.

I wonder if you could break it by just spamming a large number of print calls from 2 threads.

Again, I'm not sure, but I can't seem to find any good information on whether that stream is indeed thread-safe.

Interesting!

In standard C, there's a library function called popen, also known as "piping to a subprocess". What you get back is something like a file descriptor of a file that's been opened for input-only or output-only. (That's the nature of a pipe. It's unidirectional.) So if you pipe to the subprocess in write mode, you write to the descriptor and the subprocess reads what you've written from stdin. Or, if you pipe to the subprocess in read mode, you read from the descriptor, and the subprocess writes to stdout.

Example usage of write mode:

  • stream = popen("/usr/lib/sendmail recipient@isp.com", "w");
  • if (stream)
  • {
  • fprintf(stream, "From: %s\n", returnAddr);
  • fprintf(stream, "Reply-To: %s\n", returnAddr);
  • // etc
  • }

So you don't have to know how to talk to an e-mail server, because sendmail knows how. It's the Unix way. Many small utilities, each of which knows how to do one thing very, very well. Then string them together however you like with pipes.

Example usage of read mode:

  • stream = popen("/my/bin/decryptor < /my/encryptedfiles/file", "r");
  • if (stream)
  • {
  • // Read clear text from stream.
  • }

The parallels to cfexecute with the variable attribute are pretty obvious.

So what you're doing is a classical problem of interprocess communication, hooking up streams of data. Because you're talking about threads rather than processes, I think you're probably going to succeed at bidirectional communication.

@Roland,

That's an interesting question. I can't think of anything in the normal work flow that would require it to be natively thread-safe. Meaning, other than shenanigans like this, I can't think of a way that two things would hit the output stream at the same time.

@WebManWalking,

Piping is definitely cool stuff in the *nix world. I see all kinds of things being piped around at the command line. It's one of those features that I always see and think to myself, "Man, I wish I knew more about that."

@WebManWalking,

Ha ha, yes, I know about the command line... I mean more *understanding* how to use all the piping. I can get around in the command line, but mostly with traversal and open files. Anything beyond that and I start to crave a well-written manual :)

@Ben,

I'm really having a problem with this very thing (the output stream that's been generated so far), because of getting ready for MSIE 10. The problem is, MSIE 10 won't support conditional comments, and conditional comments are how I dealt with all the problems of quirks mode.

What I need to do is FORCE strict mode in our standard header, so that I won't need conditional comments anymore. The problem is, some of our developers have been coding their own code for the head and then calling the header routine. They shouldn't do that, but they've been able to get away with it.

As you know, any non-whitespace above the DOCTYPE causes MSIE to fall back to quirks mode. And I can't use meta tags to force strict mode until MSIE 8.

So here's what I'd LIKE to do:
(1) Grab the output buffer as you did above with getOutputStream().
(2) Parse it for stuff that must be thrown away (DOCTYPE, html tag, head tag and extraneous non-whitespace) and stuff to keep (meta tags, title tag, script tags, style tags, etc).
(3) Throw way the output buffer: (cfcontent reset="true").
(4) Generate the strict mode DOCTYPE, html and head. Insert earlier-parsed-out keepable tags appropriately (metas and title before other stuff in the head).

But I can't do that because I don't control CF Administrator, and GetPageContext() is disabled. So it's beginning to look as if I'm going to have to throw away all pre-header content (cfcontent reset="true"), without saving any of it. And that'll break some pages, maybe quite a few.

*sigh*

Running out of ideas here. Looks like I'm going to have to be the bad guy who breaks other folks' pages.

Well this is very usefull when debugging multi-threading coldfusion application. The only other alternative would be to wait until the threads are done and then output the cfthread.xxxx.output variable, which sometimes is empty (like when an error occurs).

With this method you can debug your thread in real time and see how and where a certain problem occurs.

Thank you Ben!

@Ben,

Soon I'll be implementing my bad guy scenario, doing (cfcontent reset="true") without saving the content first. In the process, I've discovered something pretty interesting (and dangerous) about calling cfcontent inside a custom tag:

It's as if the (cfcontent reset="true") is queued for execution until after the custom tag exits, then executed in the context of the page that called the custom tag. So in a page layout custom tag, if you're not careful, the entire page is thrown away!

Fortunately, like many, we don't generate the page until thisTag.ExecutionMode is "end". That's the design pattern that allows you to use thisTag.GeneratedContent. It's also what allows you to call (cfcontent reset="true") safely.

Here's the trick: If you do (cfcontent reset="true") when thisTag.ExecutionMode is "start", before generating any content, everything was fine. But if you do it when thisTag.ExecutionMode is "end", even if you do it before generating the doctype, the return to the caller will throw away the entire generated page!

Now THAT'S an undocumented pitfall!