ColdFusion QueryAppend( qOne, qTwo )

Posted July 5, 2006 at 9:26 AM by Ben Nadel

Tags: ColdFusion

Someone was asking me about a QueryAppend() method in ColdFusion. So this morning, I finally got around to writing a snippet demo about it. The way I figured it out, there are two different ways to append one query to another in ColdFusion. In the first method, QueryAppend( qOne, qTwo ), we are using the power of query of queries to union both queries together and return the resultant query. In the second method, QueryAppend2( qOne, qTwo ), we are looping through the second query and manually adding rows to the first query. As you can see from the results below, the second method (manually adding rows to the query) consistently performs much faster than the UNION ALL method.

Method One: Query of Queries

  • <cffunction name="QueryAppend" access="public" returntype="query" output="false"
  • hint="This takes two queries and appends the second one to the first one. Returns the resultant third query.">

  • <!--- Define arguments. --->
  • <cfargument name="QueryOne" type="query" required="true" />
  • <cfargument name="QueryTwo" type="query" required="true" />
  • <cfargument name="UnionAll" type="boolean" required="false" default="true" />

  • <!--- Define the local scope. --->
  • <cfset var LOCAL = StructNew() />

  • <!--- Append the second to the first. Do this by unioning the two queries. --->
  • <cfquery name="LOCAL.NewQuery" dbtype="query">
  • <!--- Select all from the first query. --->
  • (
  • SELECT
  • *
  • FROM
  • ARGUMENTS.QueryOne

  • )

  • <!--- Union the two queries together. --->
  • UNION

  • <!---
  • Check to see if we are going to care about duplicates. If we don't
  • expect duplicates then just union all.
  • --->
  • <cfif ARGUMENTS.UnionAll>
  • ALL
  • </cfif>

  • <!--- Select all from the second query. --->
  • (
  • SELECT
  • *
  • FROM
  • ARGUMENTS.QueryTwo
  • )
  • </cfquery>

  • <!--- Return the new query. --->
  • <cfreturn LOCAL.NewQuery />
  • </cffunction>

Method Two: Manually Adding Rows

  • <cffunction name="QueryAppend2" access="public" returntype="void" output="false"
  • hint="This takes two queries and appends the second one to the first one. This actually updates the first query and does not return anything.">

  • <!--- Define arguments. --->
  • <cfargument name="QueryOne" type="query" required="true" />
  • <cfargument name="QueryTwo" type="query" required="true" />

  • <!--- Define the local scope. --->
  • <cfset var LOCAL = StructNew() />

  • <!--- Get the column list (as an array for faster access. --->
  • <cfset LOCAL.Columns = ListToArray( ARGUMENTS.QueryTwo.ColumnList ) />


  • <!--- Loop over the second query. --->
  • <cfloop query="ARGUMENTS.QueryTwo">

  • <!--- Add a row to the first query. --->
  • <cfset QueryAddRow( ARGUMENTS.QueryOne ) />

  • <!--- Loop over the columns. --->
  • <cfloop index="LOCAL.Column" from="1" to="#ArrayLen( LOCAL.Columns )#" step="1">

  • <!--- Get the column name for easy access. --->
  • <cfset LOCAL.ColumnName = LOCAL.Columns[ LOCAL.Column ] />

  • <!--- Set the column value in the newly created row. --->
  • <cfset ARGUMENTS.QueryOne[ LOCAL.ColumnName ][ ARGUMENTS.QueryOne.RecordCount ] = ARGUMENTS.QueryTwo[ LOCAL.ColumnName ][ ARGUMENTS.QueryTwo.CurrentRow ] />

  • </cfloop>

  • </cfloop>

  • <!--- Return out. --->
  • <cfreturn />
  • </cffunction>

Let's explore the pros and cons of the first method.

Cons: This method performs slower for several reasons. For one, it has to evaluate SQL statements. Even though this is on the ColdFusion side, it's still a good amount of overhead. Also, due to the fact that we are unioning we have to read every row from both the first query and the second query even though we are only appending the second query.

Pros: The only real pro to this method is that we can determine wether or not we get duplicate rows back via the ALL directive in the UNION clause. This is something that cannot be done efficiently in the second method (see below).

Let's explore the pros and cons of the second method.

Cons: The main con for this method is that we are not checking for any duplicate rows. We simply append the rows from the second query to the first.

Pros: The main pro here is that its wicked fast! Much faster in fact than the first method. This will always be the case. Additionally, this behaves more like the other Append methods in ColdFusion. In StructAppend() and ArrayAppend(), you don't get a returned result object. The first object passed in is altered by reference, and therefore, no result needs to be passed back. This method of the QueryAppend() works that way, by altering the first query directly, not needing to pass anything back.

Let's explore the speed difference, as the second method will ALWAYS be faster no matter how many rows we are dealing with. Think about a situation in which you have one query that has X number of rows and you want to union it with another query which has Y number of rows.

Method One: Reads in X + Y rows in the SELECT statements of the query. Then, it writes X + Y rows back into a resultant query. That's (2X) + (2Y) read/write operations (give or take).

Method Two: Reads in just from the second query Y rows. It then appends those Y rows to the first query with Y writes. That's (2Y) read/write operations(give or take). Therefore, the second method will always be a factor of (2X) faster than the first method since it does not care about the first query in any significant way.



Reader Comments

Jun 10, 2007 at 6:56 PM // reply »
3 Comments

I'm trying your method number 2 but CF doesn't like it. Gives me this error on the line where you "set the column value on the newly created row":
An error occurred while trying to modify the query named class coldfusion.sql.QueryTable.
Query objects cannot be modified, they can only be displayed.


Jun 10, 2007 at 10:19 PM // reply »
11,238 Comments

@William,

That ColdFusion error usually comes about when you misspell one of the column names. If that happens, then ColdFusion thinks you are trying to add a new column rather than updating an existing old one.

In this case, since you are adding one query to the other, this would probably come about if the two queries do not have the exactly same column names.


Nov 18, 2008 at 1:02 AM // reply »
14 Comments

Thank you Ben Nadel!


May 26, 2009 at 5:29 PM // reply »
1 Comments

Great article. Method Two does not work when you have duplicate fields (eg disabled, datemodified, etc) when you do joins in the original queries. In SQL you distinguish the fields with the table they belong to. Therefore Method One only works.


May 26, 2009 at 5:55 PM // reply »
11,238 Comments

@Mike,

Right, but when dealing with an existing query, there is no concept of original table and therefore you should never have duplicate fields (unless your original SQL statement went wrong somewhere).

Query of queries also has this problem. Since query of queries deals with existing queries, you have no concept of originating table in the original queries.


Aug 25, 2009 at 12:52 PM // reply »
1 Comments

Hey, thanks Ben! Exactly what I needed. You rock!


Jun 22, 2010 at 11:49 AM // reply »
8 Comments

Excellent method Ben. Just been looking on how to do this and found your methods. Thanks


Apr 21, 2011 at 5:44 AM // reply »
1 Comments

As ever, exactly what I need just at this moment (you have a great presence on Google!), and a good full explanation to boot. Many thanks


Nov 28, 2011 at 2:14 PM // reply »
1 Comments

Thanks Ben for this post. It really came in handy today when I was working on a calendar application (using your calendar widget) and I had to append events from one calendar on to a second calendar in a differnet database. I did have to figure out how to invoke the function since I rarely use cfc's

<cfinvoke component="_Query_Append" method="QueryAppend2" QueryOne="#QueryOne#" QueryTwo="#QueryTwo#" returnVariable="result" >


Feb 15, 2012 at 11:04 AM // reply »
1 Comments

Thank You very much BEN


Post A Comment

Comment Etiquette: Please do not post spam. Please keep the comments on-topic. Please do not post unrelated questions or large chunks of code. And, above all, please be nice to each other - we're trying to have a good conversation here.

Please review the following issues:

Author Name:


Author Email:

Author Website:

Comment:

Supported HTML tags for formatting: <strong>bold</strong>   <em>italic</em>   <code>code</code>







  • Help Wanted - Find Your Next ColdFusion Job
Ben Nadel's Company - Epicenter Consulting Recent Blog Comments
May 21, 2013 at 7:46 PM
Using Plupload For Drag & Drop File Uploads In ColdFusion
No luck. At least I have uncovered the cause, URLScan 3.1. Here is what I see in the IIS log when a file is over 30mb. 2013-05-21 23:29:05 10.105.45.128 GET /plupload/assets/jquery/jquery-1.8. ... read »
May 21, 2013 at 6:12 PM
Using Plupload For Drag & Drop File Uploads In ColdFusion
Ben, I did not see you after Pete Freitag's Lockdown session at cfObjective but he said that IIS sets file size limits at 30MB by default which just happened to be the threshold for file size when ... read »
May 21, 2013 at 11:51 AM
Ask Ben: Parsing Very Large XML Documents In ColdFusion
Looking at my first ever XML document that I have to parse and put into MS SQL 2000 with CF8. I get it to list the desired Field name, many times over, and have a long list of this field name displa ... read »
May 21, 2013 at 9:25 AM
Turning Off and On Identity Column in SQL Server
you are awesome..i am lucky to get this blog between such a garbage one....Thanks, Prashant ... read »
May 20, 2013 at 4:38 PM
Using A Dynamic Column Name With ValueList() In ColdFusion
@Dana, Your confusion is well founded, since this is a very confusing features. In fact, it ONLY works if you use array notation. Meaning, that this: arrayToList( query[ "columnName" ] ) ... read »
May 20, 2013 at 4:34 PM
Using A Dynamic Column Name With ValueList() In ColdFusion
I was thinking chicken and the egg, I wouldn't have expected it to work in the valuelist going in I guess. Maybe I just need a beer, long day :) ... read »
May 20, 2013 at 4:29 PM
Using A Dynamic Column Name With ValueList() In ColdFusion
@Dana, That's if you're trying to reference a specific row. In this case, we're trying to reference the entire query column as one cohesive value. So, you are correct that if you wanted to output a ... read »
May 20, 2013 at 4:24 PM
Using A Dynamic Column Name With ValueList() In ColdFusion
I thought when you used array notation to reference queries you always had to have the row or it would throw a similar error as well? ... read »
InVision App - Prototyping Made Beautiful With Prototyping Tools