Skip to main content
Ben Nadel at cf.Objective() 2013 (Bloomington, MN) with: Brian Rinaldi
Ben Nadel at cf.Objective() 2013 (Bloomington, MN) with: Brian Rinaldi

Normalizing Query Cross Join Data Using Namespaces In ColdFusion

By
Published in , Comments (7)

For simple CRUD (Create, Read, Update, Delete) queries, I'm usually working with a single database record at a time. But, for user interfaces (UIs) that require more "reporting" style data, I'll often use a more complex, cross join SQL query in order to avoid N+1 query problems. When using JOINs in this manner, I like to namespace the data from each table using a {table}_ prefix. Then, when collating the data back on the ColdFusion side, I'll normalize this data back into separate objects. Historically, this has been a very manual process. I'm trying to find ways to make this a little less manual.

In Big Sexy Poems, I have several "list" pages that pull data from multiple tables. Consider the page that shows the list of "share links" across all poems written by the given author. The SQL for that query might look like this:

SELECT
	-- Poem data.
	p.id        AS poem_id
	p.name      AS poem_name

	-- Share data.
	s.id        AS share_id
	s.name      AS share_name
	s.viewCount AS share_viewCount
FROM
	poem p
INNER JOIN
	share s
ON
	(
			p.userID = :userID
		AND
			s.poemID = p.id
	)

As you can see, the data being read out of the poem table is prefixed with poem_ and the data being read out of the share table is prefixed with share_.

In recent years, I've [mostly] stopped using the ColdFusion query object; and, instead, return arrays from all of my CFQuery tags. Arrays-of-structs map more seamlessly onto my preference for data structures; and, onto my love of .map(), .filter(), and .each() member methods. As such, let's assume that the above query execution results in an array of structs - structs whose keys contain name-spaced data.

What I want to do is perform an in-place transformation of each struct (within the results array), moving each key from its name-spaced location into a sub-struct location. Essentially, any key that starts with poem_ should be moved into a sub-struct named poem; and, any key that starts with share_ should be moved into a sub-struct named share.

Aside: Isn't mutating data in-place a huge "party foul"? No, transforming data as it comes out of the database is an implementation detail of the data-access layer (DAL). As such, how you implement said transformation is of no one else's concern.

With that said, here's what I've come up with:

<cfscript>

	// Consider data coming back from the database using an INNER JOIN. It will create a
	// cross-product between the two tables. I usually "namespace" each column using an
	// underscore (e.g. "poem_id" and "share_id").
	/*
		SELECT
			p.id        AS poem_id
			p.name      AS poem_name
			s.id        AS share_id
			s.name      AS share_name
			s.viewCount AS share_viewCount
		FROM
			poem p
		INNER JOIN
			share s
		...
	*/
	queryData = [
		{
			poem_id: 1,
			poem_name: "First poem",
			share_id: 101,
			share_name: "Public link",
			share_viewCount: 4
		},
		{
			poem_id: 1,
			poem_name: "First poem",
			share_id: 102,
			share_name: "Family chat",
			share_viewCount: 9
		},
		{
			poem_id: 2,
			poem_name: "Second poem",
			share_id: 203,
			share_name: "Blog demo",
			share_viewCount: 25
		},
		{
			poem_id: 3,
			poem_name: "Third poem",
			share_id: 304,
			share_name: "Public link",
			share_viewCount: 8
		},
	];

	// Transform the cross-product data into normalized structs.
	writeDump( normalizeCrossJoin( queryData ) );

	// ------------------------------------------------------------------------------- //
	// ------------------------------------------------------------------------------- //

	/**
	* I split each row (within a cross-join) into a collection of separate objects. This
	* transformation is performed in-place.
	*/
	private array function normalizeCrossJoin( required array results ) {

		return results.each( ( row ) => normalizeCrossJoinNamespaces( row ) )
			?: results // Hack because ACF doesn't return a collection from .each().
		;

	}

	/**
	* I split the given cross-product struct into multiple structs. The name of each
	* sub-struct is determined by the first element in the delimited list. This
	* transformation is performed in-place.
	*/
	private void function normalizeCrossJoinNamespaces( required struct input ) {

		for ( var key in input.keyArray() ) {

			// Expects most entries to be in the form of `{subName}_{subKey}`.
			var parts = key.listToArray( "_" );

			// If there's no namespace, or the delimiter occurs too many times, skip over
			// the key - it's unclear how to process it.
			if ( parts.len() != 2 ) {

				continue;

			}

			var subName = parts[ 1 ];
			var subKey = parts[ 2 ];
			// MOVE KEY into sub-key. Note that ColdFusion will create a new struct as-
			// needed. We don't have to perform any special check for the namespace.
			input[ subName ][ subKey ] = input[ key ];
			input.delete( key );

		}

	}

</cfscript>

When I run this Adobe ColdFusion code, I get the following output:

CFDump of an array of struct where each struct is composed of two sub-structures.

As you can see, all keys starting with poem_ have been moved into a sub-structure named poem; and, all keys starting with share_ have been moved into a sub-structure named share.

This transformation is partly for convenience - the data becomes a little easier to work with, especially if further collation and aggregation need to be performed. And, to be fair, this is partly for aesthetics - outputting entry.poem.name is just objectively nicer looking than outputting entry.poem_name. And, for me, how the code looks is just as important as how the code works.

Want to use code from this post? Check out the license.

Reader Comments

38 Comments

My brain always starts with the data and the database, So I cringe when I look at database tables that don't prefix the column names with the table name.
In a SQL select statement:

p.poem_id, p.poem_name, s.share_id, s.share_name

Looks much more clear to me than:

p.id, p.name, s.id, s.name

So what you accomplish with the aliases in your example select statement, I much prefer to see that as part of the table schema so the data is always clearly identified as to which table it belongs to in the database.

For your statement:

And, to be fair, this is partly for aesthetics - outputting entry.poem.name is just objectively nicer looking than outputting entry.poem_name. And, for me, how the code looks is just as important as how the code works.

I agree with this; if you're only going to manipulate that data using the structures then there is consistency and cleanliness. Especially knowing that you can always guess that most of the structs will have a id and name key without having to go back and double check things.

I'm not sure I had much of a point to this comment, I guess I just wanted to give you some engagement to encourage you to keep doing what your doing and let you know I appreciate your sharing of your process with us!

16,109 Comments

@Ken,

I appreciate the kind words 🙂

Like you, I basically always start with the database schema. I like to get it straight in my head how things are modeled and how they relate to each other. I like to think about the indexes that make sense for performance; and the indexes that makes sense for constraints. Database are just fun, AMIRIGHT?!

In my career, I've worked at places that each do something different:

  • Don't prefix any of the columns.
  • Prefix only the id column (ex, poem_id, name).
  • Prefix every column (ex, poem_id, poem_name).

Over time, I've just gravitated towards the "prefix nothing" camp. I think, in general, I find it a little more readable. But, to be fair, I'd say that in like 95% of all cases, I'm only ever reading my own code since I've never worked on a large team. I can see how having the columns prefixed when reading other people's code could be very nice.

I also know that as I grow older, more and more of all of this is wildly subjective. I feel like half of becoming an adult is just learning that everything you thought was "objective" is — to quote The Big Lebowski"yeah, well, you know that's just like your opinion, man". 😆

1 Comments

Ben,

You know I'm always preaching the "Let the DB do what it's great at". Rather than doing all this transformation in code, why not get the resulting JSON you want right away from the DB? Most major DBMSs now support full relational to JSON native transformation. For example, in MySQL you can do the following:

SELECT
  COALESCE(
    JSON_ARRAYAGG(
      JSON_OBJECT(
        'poem',  JSON_OBJECT(
          'id',   p.id,
          'name', p.name
        ),
        'share', JSON_OBJECT(
          'id',        s.id,
          'name',      s.name,
          'viewCount', s.viewCount
        )
      )
      ORDER BY p.id, s.id
    ),
    JSON_ARRAY()
  ) AS data
FROM 
  poem AS p
  JOIN share AS s
    ON s.poemID = p.id
WHERE 
  p.userID = :userID;

this will give you the result needed in the format:

[
  { "poem": { "id": 1, "name": "First poem" }, "share": { "id": 101, "name": "Public link", "viewCount": 4 } },
  { "poem": { "id": 1, "name": "First poem" }, "share": { "id": 102, "name": "Family chat", "viewCount": 9 } },
  { "poem": { "id": 2, "name": "Second poem" }, "share": { "id": 203, "name": "Blog demo", "viewCount": 25 } },
  { "poem": { "id": 3, "name": "Third poem"  }, "share": { "id": 304, "name": "Public link", "viewCount": 8 } }
]
16,109 Comments

@Phill,

Ha ha, I was literally just considering this over on LinkedIn a few minutes ago. It's a really fascinating idea. My only concern would be some of the added complexity. But, actually seeing you write it out, it's not as complex as I had made it out in my head.

I'd even be happy getting rid of the COALESCE() and the JSON_ARRAYAGG() calls and just having a helper method on the ColdFusion side that iterates and calls deserializeJson(). That would make the SQL even more readable with still only minimal ColdFusion work.

4 Comments

It never ceases to amaze me how much our cfml journeys are parallel. It's almost creepy, and since you started blogging I felt that I never needed to mostly, as you already say what I would have posted anyway, from the time you started until now. That said there is much I should have blogged but didn't due to this also. So thanks for saving me time #lol

Was just extending my dal and moving to object-based queries like this in my teamcfml framework dal module. It also helps returning api data in graph-like ways in an rest-api also.

Cheers again Mr Nadel.

4 Comments

IMHO you need both techniques, as quite often manipulating data from different sources (db,clickhouse,social media, apis) to combine into one useful data object for presentation layer also.

That said i'm enjoying sql 2025 with polybase, and the ease that combination brings from ETL by allowing you to query sql and excel/csv files in the same query without having to import into the db first!

So yeah, bringing data from all kinds of places to be useful in queries adds complexity, but my dal takes that all away.

just call tc.dal.getNestedPagePoemShares(2,20);

16,109 Comments

@Dawesi,

What can I say - clearly great minds think alike 🙌

Pulling in data from multiple sources is a whole all ball of complexity. Sometimes, I really want to find a nice abstraction that makes zipping-together code more easy; but, I struggle to find the "right" abstraction; or one with a nice API design. And then sometimes, I think the abstraction can even make the code harder to read because it's less obvious what the code is doing.

It's an ongoing battle to balance DRY code, complexity, and readability. But, I keep fighting!!

Post A Comment — I'd Love To Hear From You!

Post a Comment

I believe in love. I believe in compassion. I believe in human rights. I believe that we can afford to give more of these gifts to the world around us because it costs us nothing to be decent and kind and understanding. And, I want you to know that when you land on this site, you are accepted for who you are, no matter how you identify, what truths you live, or whatever kind of goofy shit makes you feel alive! Rock on with your bad self!
Ben Nadel
Managed hosting services provided by:
xByte Cloud Logo