Skip to main content
Ben Nadel at Scotch On The Rock (SOTR) 2010 (Munich) with: Jens Hoppe
Ben Nadel at Scotch On The Rock (SOTR) 2010 (Munich) with: Jens Hoppe

Claude Code Convinced Me To Simplify My SQL JOINs

By
Published in , , Comments (17)

For the last 20 years, I've had very strong feelings about how you should structure your SQL JOIN clauses. In my mind, a SQL statement is meant to be read from top to bottom. Which historically meant that each FROM, INNER JOIN, and ON clause described the "path" that the query should take when looking for the desired records. In essence, each ON clause described both the relationship between the two tables and the filtering that should be performed as part of that relationship.

To be clear, this had nothing to do with how SQL works under the hood. For everything that I do, the query optimizer is already doing the same thing. My SQL structure was never about fixing the query performance — it was about telling a human story.

As an illustration, consider when the missus sends me to the supermarket to pick up "Quaker Oatmeal" (num-nums!). As I'm walking through the store, I'm doing many things simultaneously:

  1. Looking for the correct isle.
  2. Looking for the oatmeal section.
  3. Selecting the Quaker brand.

To transpose this human perspective onto a SQL statement, it would look like this:

FROM
	isle i
INNER JOIN
	product p
ON
	(
			i.number = '3' -- The isle I'm in.
		AND
			p.isleID = i.id -- The products in THIS isle.
		AND
			p.type = 'Oatmeal' -- Am I looking at the right product?
		AND
			p.brand = 'Quaker' -- Am I looking at the right brand?
	)

This is how I think about the world. And databases are all about mapping the world onto a digital persistence layer. So it always made sense for my SQL structure to mimic the way my brain works.

As a human man with a human man brain, I do not:

  1. Pick up every product in the store.
  2. Put down every product that isn't:
    • The correct isle.
    • The correct type.
    • The correct brand.

And yet, this mentality is transposed onto how most SQL statements are structured:

FROM
	isle i
INNER JOIN
	product p
ON
	p.isleID = i.id -- Pick UP EVERY PRODUCT in the store.

-- ... then once I have EVERY SINGLE PRODUCT in my basket...

WHERE
	i.number = '3' -- Put DOWN products in other isles.
AND
	p.type = 'Oatmeal' -- Put DOWN products of other types.
AND
	p.brand = 'Quaker' -- Put DOWN products of other brands.

Again, to be clear, these two approaches are functionally equivalent. The SQL database is doing this work for you behind the scenes. Both SQL statements do the same work, use the same optimizations, leverage the same indices and yield the same result.

The difference is only in how the SQL statement itself "tells the story". And Claude Code finally convinced me to get on board with using the latter approach.

How Claude Code Changed My Mind

At work, we do a lot of "vibe coding". But in my personal life, Claude Code is mostly here in an advisory capacity. I do my deep thinking, I execute the work, and then (sometimes) I ask Claude if there are ways for me to improve my technique.

Most of the time, Claude Code doesn't have many suggestions. Much of the time, the suggestions that it does have are not very persuasive. But one suggestion that it refused to give up on was simplifying my SQL statement structuring.

For the SQL joins, it made several compelling arguments:

  1. Most people don't actually think about the SQL.

  2. Most people don't actually read the SQL from top to bottom.

  3. The SQL clauses provide a separation of concerns:

    • The JOIN describes how the tables are related.
    • The WHERE describes how the rows are filtered.
  4. Isolating filtering in the WHERE clause makes SQL easier to debug because it simplifies the ability to remove all filtering and then incrementally re-add filtering.

  5. LEFT OUTER JOIN relationships often have to move filtering into the WHERE clause anyway (or subtle bugs can arise).

Arguments 1 and 2 are mostly about how much people are susceptible to wearing blinkers; and about how much they rely on momentum and muscle memory to get the work done without thinking too deeply. For my personal work, this isn't that relevant. However, SQL is something that I use both personally and professionally. As such, there is absolutely value in keeping techniques consistent if it means working with the grain of how so many other people think.

Arguments 3 and 4, though, were really what sold me. It forced me to stop and think about debugging complex SQL queries in the past; and, about the friction of going to a number of different ON clauses and then commenting-out just part of the condition.

And as much as I do still believe that a compound ON statement better aligns with my "human thinking", I'm now convinced that separating "joining" and "filtering" into two distinct areas of the SQL statement has a superior value.

And even as I was still wavering on the balance of it, the 5th argument was what removed any remaining doubt. As much as I loved having all my filtering collocated in one place, using a LEFT OUTER JOIN often required moving some filtering to the WHERE clause such that it (the filtering) was applied to the resultant rows and not to the table relationship.

By treating the WHERE clause as the one place that filtering would be applied, it meant that both INNER JOIN and LEFT OUTER JOIN relationships would be treated mechanically the same. And dang it if I don't love me some uniformity!

I'm Not Crazy, I Just Need To Be Convinced

I'm not crazy; I'm just never convinced by the "this is how we've always done it" argument. I need you to point to the doll and show me exactly how your decisions helped you. I'm a reasonable person and I need reasonable arguments. And that is exactly what Claude Code did — it gave me reasonable arguments that were stronger than my reasonable arguments.

Want to use code from this post? Check out the license.

Reader Comments

312 Comments

I've always done my filtering as Claude suggests. It's just how my brain makes sense of SQL, so it's nice to know I fell on the right side of this equation.

I've also found AI really helpful in discovering other approaches to SQL design and optimization. Approaches I was never aware of before, so it's been a useful coach for me in this regard.

16,233 Comments

@Chris,

I think it's how most people think, so you're in good company. I think I can't hold too many things in my head at once; so there was always something very comforting about being able to see both the relationships and the filtering right in the same place. It gave me a very tightly scoped bit of information to wrap my head around.

I'm also trying to be good / better about taking tips and pointers from Claude / AI. It's been a bit hit-and-miss, though. Sometimes it will make suggested changes. But then when I pressure-test those changes, asking for more clarity and reasoning, it will often fess-up that it's suggestions were not as strong as they were originally stated to be.

That's what made this suggestion so interesting — Claude refused to back down. It kept asserting that this was the right approach to use.

7 Comments

But is it as efficient?

I know that is always difficult to know and justify. Around 14 years ago, I found Navicat and I started doing all SQL development on my computer. Once I was able to return the data I wanted I would change 'how and where' the JOINs and ONs were written. I also worked on 'what and where' I put in the WHEREs.

I would run each change and note the time. Come cases returned times so close I would keep ALL the wheres in the WHERE clause.

Rare cases returned the opposite and would chose the one that was easiest to modify. I cared less about understanding why and more about getting the code written and the application.

Also to note, if the same lookup was used in another page I moved it to a stored procedure in the database engine.

16,233 Comments

@Gary,

First of all, +1000 on Navicat! It's been my favorite SQL GUI for like 15 years. It's just a super solid product. I've even used it a bit for MongoDB... and I think they may even have a Redis GUI as well. But, I'm in the SQL biz like 99.99% of the time.

That said, it's been a while since I've worked on anything where the SQL gets too tricky. These day, most of DB reads are either ID-based look-ups for mutation or view-specific JOIN for optimization. It's the latter that only ever brings this to light. And for those, I run an EXPLAIN locally (in dev) to make sure it's not crazy; and then I kind of just "vibe it out" in production to make sure the performance feels right.

When I was in a more high-volume environment at InVision, I would often use Fusion Reactor (application performance monitoring tools) and the MySQL slow log to keep track of slow running requests. Then, when things showed up, I would address them at that time.

For the most part, the dev/prod divide worked "the same" like 99% of the time. But, there was definitely that 1% case where the query optimizer just went on some crazy acid-trip once the SQL got deployed. 😨

16,233 Comments

@Carl,

The LEFT OUTER JOIN was one of the things that finally sold me on moving more conditions to the WHERE clause. Depending on what the condition is doing / what your needs are, the LOJ can accidentally remove rows from the results.

For anyone interested in this point, consider a user table with an optional timezone table. If one were to do:

FROM
	user u
LEFT OUTER JOIN
	timezone t
ON
	(
			u.id = ?
		AND
			t.userID = u.id -- FAIL (no matching row)
	)

... if there is no corresponding t row, then I think (don't have a DB running to test this) it will end up returning all users since the condition fails and nothing is there to limit the left-hand selection.

However, if you move the u.id condition to the WHERE:

FROM
	user u
LEFT OUTER JOIN
	timezone t
ON
	t.userID = u.id -- FAIL (no matching row)
WHERE
	u.id = ?

... then you still get the u.id=? in the WHERE even though the JOIN fails. Which means you still only get one record.

So, by moving all the filter conditions into the WHERE, you basically never need to worry about this.

Caveat: I do think it matters if you want the right-hand side of the result to be NULL for certain things, though. If you want the right-hand side to be NULL, you have to move conditions into the ON to break the cross-product.

17 Comments

Hey Ben,

I find Claude's declaration that "Most people don't actually think about the SQL" to be the honest truth.

For example, most devs (not all) have no desire to write database procedures, no matter what language they use. Even knowing performance would improve.

Instead they may use an ORM which abstracts SQL, or in the CFML world, a CFQuery.

The typical wisdom is "it's easier to modify" as Gary mentioned.

Generally, I think agents will help improve everyone's thinking. Like you did, just ask the agent and try to keep an open mind.

7 Comments

@Noursecode, This is all very interesting. I'm a good CF developer which led me to be a better SQL developer. I also learned SQL from a few great database masters.

I have not used Claude or any Advanced Instruction system because I don't like the marketing types implying that AI is Ai. That's on me and I am going look at using Claude. That is, after all, how we evolve in this industry. This has caused me to think back about how I came to this point in my life.

I became interested in electronics when I was eight years old. My father was an Air Force aircraft radio operator back in the dark ages and got his HAM licenss long before I was born. Our house was filed with decommissioned Air Force radio equipment. Those radios led me to electronics which led me to computers which led me hardware which led me to software which led me to the U.S. Army in the early 70s.

A lot happened in that period and a lot more happened in the next periods of my life-in which one thing always held true-that kept me going. I was a natural with this and I could visualize it in my mind.

It was all fun and games and that sometimes got me in trouble. I shy away from this AI crap because it reminds me of the old chat games of the 80s. They didn't end well, yet they're back. This time it might be different and maybe this time it's okay to let fun into work.

Thank you, Ben.

16,233 Comments

@Gary,

So far, I haven't really used AI to do anything that I couldn't already do (with the exception of one or two experiments that I've already lost interest in). What I'm hoping to use it for mostly is to allow me to take larger steps outside of my comfort zone.

For example, I've never played with full-text indexing. This site has no search. Would people use search anyway? Probably not. I actually just use site:bennadel.com {query} in Google when searching the site. But, with AI, I feel more confident that I could get full-text indexing working with a new technology (like Elastic Search or something).

7 Comments

@Ben Nadel, Something else that people don't understand. The current AI does not do anything that someone hasn't already done. AI just does it quicker.

I want to compare to models. I can already do so but it is time consuming. Or I just ask AI to do it. I am going to ask Claude to create a simple search page using ColdFusion.

I know i need to start using Lucie and I promise I will on my new site,

16,233 Comments

@Gary,

It's a fascinating point. We're wildly off-topic at this point, but who cares 😂 I see a lot of people talking about how easy it is to just rebuild some prior art. One-shotting some existing SaaS product or converting a library from one language to another. The impressive bit there is obviously in the scope of what's being done (and the shocking degree to which it "just works").

But, in my experience, what's really hard isn't the coding — it's the talking to the client, understanding what they're asking for, trying to find the right UX to suite their needs, and then iterating with them day-after-day to arrive at the right solution.

There's no way to "one shot" what's in someone else's head!

7 Comments

@Ben Nadel, Yes, sorry, my fault, but that's how conversations go and it's been years since we've had a conversation. I'm a B-string programmer and I follow you A-string guys. I converse with you guys when I find the opportunity,

Back to SQL, I have often used a Query of Queries to filter data. There are times when you just can't get what you want in one query by filtering in the Where statement.

16,233 Comments

@Gary,

Ha ha, I wasn't complaining at all! Tangential conversations are the spice of life 😂

As far as query of queries goes, I've taken to returning arrays from all of my data access methods. And then I can do the filter and map methods on those arrays. Not better or worse, just coming at it from a parallel angle that works well for me. It's so much fun to think about this stuff!

16,233 Comments

@Gary,

It is super interesting! And it's great that they were able to move past a vendor that was clearly treating them terribly. But two things do pop to mind. 1) there is something kind of "icky" about the notion of screen-shotting someone else' work and just saying "copy that". I totally get that they were between a rock-and-a-hard-place; but, the idea that our industry is now all OK just copy-pasting other people's work doesn't sit great - it's a brave new world. And 2) they now have to maintain this software going forward.

So, on the one hand, it's bonkers how good something of this tooling is getting. But on the other hand, I always think at what cost.

7 Comments

@Ben Nadel,

Your comment reminds me of a discussion I had with the team at Boardwatch Magazine in the mid 90s. We had just returned from a very successful ISPCON and were talking about the state of the Internet. I listened to what everyone had to say then Jack looked and me and said, "Garheart, what do you have to say?"

"Well, you know," I replied, "the good thing about the Internet is, everyone can be on it. The bad thing about the Internet is, everyone can be on it."

That pretty much sums up every major advancement given to the masses, No matter what it is, no matter to good it brings, some will use it poorly.

Post A Comment — I'd Love To Hear From You!

Post a Comment

I believe in love. I believe in compassion. I believe in human rights. I believe that we can afford to give more of these gifts to the world around us because it costs us nothing to be decent and kind and understanding. And, I want you to know that when you land on this site, you are accepted for who you are, no matter how you identify, what truths you live, or whatever kind of goofy shit makes you feel alive! Rock on with your bad self!
Ben Nadel
Managed ColdFusion hosting services provided by:
xByte Cloud Logo