Skip to main content
Ben Nadel at cf.Objective() 2010 (Minneapolis, MN) with: Bob Silverberg
Ben Nadel at cf.Objective() 2010 (Minneapolis, MN) with: Bob Silverberg

You Can Modify Elements During Filtering In ColdFusion

By
Published in Comments (12)

I have a mental hurdle to overcome: every time I modify an element during an array .filter() operation, I feel dirty. I feel like I've violated some sort of semantic contract. But, this contract is an entirely self-imposed constraint. The problem is, words have weight. And the fact that the method is called "filter" has allowed my ignorant little caveman brain to believe that filtering is the only thing this method should be doing. I need to grow-up and stop this nonsense.

Consider this ColdFusion code that filters an array of users and modifies the collection at the same time by computing a name property:

<cfscript>

	users = [
		{ firstName: "Anna", lastName: "Banana" },
		{ firstName: "Blakely", lastName: "Smith" },
		{ firstName: "", lastName: "" },
		{ firstName: "Donkers", lastName: "Vaughn" }
	];

	filteredUsers = users.filter(
		( user ) => {

			// MODIFY the element during filtering.
			user.name = "#user.firstName# #user.lastName#";

			// FILTER the element into the results.
			return (
				user.firstName.len() &&
				user.lastName.len()
			);

		}
	);

	writeDump( filteredUsers );

</cfscript>

The .filter() operator is performing two duties:

  1. It's computing the name property via firstName and lastName concatenation.

  2. It's returning a Boolean result, dictating whether or not the element should be included in the subsequent array.

This is 100% totally fine! I have to stop feeling guilty about imaginary issues. The .filter() method is still being used to "filter". It's a tool; and I'm using that tool to make the program both simple and expressive.

Consider the following collection methods in ColdFusion:

  • .filter()
  • .map()
  • .each()
  • .reduce()

These methods are all iterators. But, they have different semantics. The trap that I keep falling into is that I think those semantics refer to what logic is valid within the operator function. But, the semantics have nothing to do with what happens inside the operator — the semantics solely refer to:

  1. How the result of each operation is consumed.

  2. How the overall result of the iteration is reported.

That's it. That's the entirety of the semantics. Within the implementation details of the iteration function, I can do whatever my program needs me to do. I have to stop limiting myself based on silly constraints.

Ben, be better!

Want to use code from this post? Check out the license.

Reader Comments

17 Comments

I dunno... the fact that the modification affects both the original Users and the listedUsers arrays, but the filter only affects the newly created listUsers array just seems... wrong... to MY ignorant little caveman brain.

16,109 Comments

@Andrew, yeah, sorry, I think you can only edit a post for like 6-hours after it was posted. I can increase that - it's all just a balancing act.

The way I look at it, the different iteration techniques are all about the mechanics of what the loop is doing. For example, the difference between map() and flatMap() is about how the result of the iterator is handled. But, it doesn't imply too much about what is actually going on during the iteration.

You could also always just overwrite the original variable, ala:

users = users.filter( ... )

When you do that, I think it starts to get even more interesting, because you could argue that the filter, in this case, is almost acting like a map as well.

16,109 Comments

Here's an interesting thought experiment, how might one go about recording the number of items that are filtered-out for various reasons. I could have code that looks like this:

<cfscript>

	missingFirstName = 0;
	missingLastName = 0;

	filteredUsers = users.filter(
		( user ) => {

			if ( ! user.firstName.len() ) {

				missingFirstName++;
				return false;

			}

			if ( ! user.lastName.len() ) {

				missingLastName++;
				return false;

			}

			return true;

		}
	);

</cfscript>

Here, the filter() method is both filtering the array and mutating shared state. In this case, the filter operator is doing more than just filtering the array; so, is this wrong?

22 Comments

Hallo mate. I think you're missing the whole "use the right too for the job" concept, and also forgetting... you're not the only person who will be reading your code.

map, filter, reduce etc have meanings as concepts, so when someone else sees your code and see that it filters (BECAUSE YOU ARE CALLING FILTER), they will expect it to a) filter; b) not have side effects; c) not also kinda do a map. Your code is breaking the principle of least astonishment, which is... not as good as it could be.

In yours first example you could use less "astonishing" code by taking the time to understand that the task at hand is to filter out some records, and remap the good ones:

namedUsers = users
    .filter((user) => user.firstName.len() && user.lastName.len())
    .map((user) => user.insert("name", "#user.firstName# #user.lastName#"))

In your second example, yer using input users to do two things: remap the good ones, and tally the bad ones. This is a reduction to me: translate one data structure into another.

namedUsersWithMetrics = users.reduce((metrics, user) => {
    if (user.firstName.len() && user.lastName.len()) {
        user.name = "#user.firstName# #user.lastName#"

        metrics.users.append(user)
        return metrics
    }

    if (!user.firstName.len()) {
        metrics.missingFirstName++
    }
    if (!user.lastName.len()) {
        metrics.missingLastName++
    }

    return metrics
}, {
    missingFirstName  = 0,
    missingLastName = 0,
    users = []
})

(this also fixes the logic error in your example).

You are taking yer hammer and going "ooh look! A Nail!". But if you thought about things some more, you'd see it's not a nail. And it's also not a hammer.

I think you could do with applying some "developer humility" to your approach to these things. When there's an industry-adopted principle, and you think you see where it's wrong or could be improved by doing things "The Nadel Way"... you're probably mistaken because these principles almost certainly exist for a reason and were arrived at by ppl cleverer than you or me.

--
Adam

Code: https://trycf.com/gist/df955c09f7960812c42862714fa4c076/acf2023?theme=monokai

16,109 Comments

The reduce() function is the one iteration function that I really do want to love; but, when I finally use it, in 90% of cases, I look at the code and feel like it never reads as well as a for loop would read. I think a lot of that comes from the initial value being defined below the logic of the loop. I often wonder if I would use .reduce() more if the initializer value was the first argument, not the second. So, takine your rewrite, I would probably rewrite it as this:

missingFirstName  = 0;
missingLastName = 0;
selectedUsers = [];

for ( user in users ) {

    if ( ! user.firstName.len() ) {
        missingFirstName++;
        continue;
    }

    if ( ! user.lastName.len() ) {
        missingLastName++;
        continue;
    }

    user.name = "#user.firstName# #user.lastName#";
    selectedUsers.append( user );

}

There's just something about seeing the base values at the top that my brain really craves.

17 Comments

Ben - but you can see the base values at the top with reduce or am I misunderstanding?

namedUsersWithMetrics = users.reduce( ( metrics={ missingFirstName = 0, missingLastName = 0, users = [] }, user ) => {
	if (user.firstName.len() && user.lastName.len()) {
		user.name = "#user.firstName# #user.lastName#"

		metrics.users.append(user)
		return metrics
	}

	if (!user.firstName.len()) {
		metrics.missingFirstName++
	}
	if (!user.lastName.len()) {
		metrics.missingLastName++
	}

	return metrics
} );
16,109 Comments

@Andrew,

That's an interesting idea! And to be honest, I'm not sure what ColdFusion will do at this point. From the docs, it looks like you can pass null (ie, omit) for the initial value. But, I'm not sure what that results in. I know in some languages (?JavaScript?) omitting the initial value will cause the first value in the collection to be used as the initial value (and then I think the iteration skips over the first value implicitly). If ColdFusion does that, then you won't get your fallback argument.

Very thought provoking. I'll try it out in the morning just to see what happens.

17 Comments

You need to be aware of situations where you attempt to reduce an empty array. In Adam's example with the trailing default, namedUsersWithMetrics will equal the default when the array is empty. However, in my example, where the default is set up top, namedUsersWithMetrics will be NULL.

22 Comments

I think a lot of that comes from the initial value being defined below the logic of the loop

So... init it first! Blimey.

Building on what I had before:
https://trycf.com/gist/6bdd830c62fe795f9a394061a29822b4/acf2023?theme=monokai

namedUsersWithMetrics = (()=>{
    var initialState = {
        missingFirstName  = 0,
        missingLastName = 0,
        users = []
    }
    return users.reduce((metrics, user) => {
        user.firstName.len() ?: metrics.missingFirstName++
        user.lastName.len() ?: metrics.missingLastName++

        if (user.firstName.len() && user.lastName.len()) {
            metrics.users.append(user.insert("name", "#user.firstName# #user.lastName#"))
        }

        return metrics
    }, initialState)
})()

One doesn't need the IIFE here, but I figured it was a nice thought when creating what amounts to be a throw-away variable.

90% of cases, I look at the code and feel like it never reads as well as a for loop would read

Indeed. But this is not the point we're making here.

The point is "don't use filter for something that is not a filter operation. In this case your operation is translating one data struct to another, so... in higher-order-function situations: it's not a filter, it's a reduce. My code example demonstrates how to do the reduce operation, in a HOF-semantic way.

Would I use the code above? probably not: I'd use a for loop.

But I would use a for loop. Not a filter. It's not a filter.

1 Comments

Your post on such an interesting subject has left me speechless. I regularly check out your blogs and stay current by reading the material that you offer; nevertheless, the blog that you have posted today is the one that I appreciate the most.

16,109 Comments

So, in a stroke of irony, I actually had to change the strategy in my code due to a bug in Adobe ColdFusion. It turns out, if you use async iteration on an array, and then the operator you're using also performs array iteration, the closed-over variables suddenly disappear:

www.bennadel.com/blog/4831-adobe-coldfusion-bug-nested-array-iteration-breaks-closure-variables.htm

So, in the end, I had to change this "offending" code:

return gateway
 	.makeRequest( resource, searchParams )
 	.filter( ( result ) => normalizeResult( result ) )
;

... where the .filter() call was both filtering and mutating, into this:

var results = gateway.makeRequest( resource, searchParams );
var filteredResults = [];

for ( var result in results ) {

	if ( normalizeResult( result ) ) {

		filteredResults.append( result );

	}

}

return filteredResults;

This code is doing the exact same thing, with without the .filter() call. And, it allows me to work around the aforementioned Adobe ColdFusion bug.

Post A Comment — I'd Love To Hear From You!

Post a Comment

I believe in love. I believe in compassion. I believe in human rights. I believe that we can afford to give more of these gifts to the world around us because it costs us nothing to be decent and kind and understanding. And, I want you to know that when you land on this site, you are accepted for who you are, no matter how you identify, what truths you live, or whatever kind of goofy shit makes you feel alive! Rock on with your bad self!
Ben Nadel
Managed hosting services provided by:
xByte Cloud Logo