Thoughts On Chained And Dependent Algorithm Steps
Not all the time, but enough that it has pained me, I have had an algorithm that has a number of steps in which each step depends on the successful completion of the previous step. Now, I know what you are saying - doesn't every algorithm work that way? I guess, to some degree, yes; but, sometimes, it just feels more obvious. For example, I was working on a data import process yesterday that has the following steps:
- Upload Excel file
- Make sure it is upload file in an Excel document
- Convert Excel to query using POIUtility.cfc
- Move the query data to the database
Here, we have multiple chances to throw an error; and, not all of these errors would be totally unexpected, which is what makes this scenario seem so interesting to me. Currently, to deal with this kind of chained logical step model, I have some sort of Error object or array that helps me keep track of the success of the algorithm. The pseudo code might look something like this:
- Perform step 1
- IF no errors, perform step 2
- IF no errors, perform step 3
- ....
- IF no errors, perform step N
- IF no errors SUCCESS
- ELSE display errors
This works, but what I don't like about it is the IF statements that have to go around each subsequent steps. I hate having to constantly check to see if an error has previously been thrown. Something about it just seems less than clean. Maybe it is, maybe it's not - just a (strong) gut feeling.
Anyway, I thought it would be cool if there was some sort of chained logical tag system that would allow this interdependence of step success to take place without IF statements; something where, if Step 2 failed, it would automatically know NOT to allow steps 3->N to process. I put some ideas down on paper. This is mostly pseudo code, so please forgive any syntactical errors (as I cannot run this):
<!--- Create an array to hold processing errors. --->
<cfset arrErrors = [] />
<!---
Process the data in a multi-step dependent algorithm
where each step depends on the successful completion
of all the previou steps.
--->
<chain:steps>
<!---
For the first step, we are going to be uploading
the selected form file.
--->
<chain:try>
<cffile
action="upload"
filefield="resume"
destination="#REQUEST.UploadDirectory#"
nameconflict="makeunique"
/>
<!--- Make sure this is a CSV file. --->
<cfif (ListLast( CFFILE.ServerFile, "." ) NEQ "csv")>
<!---
This was not a valid file extension for
this process. Throw an error (which will be
caught by the internal catch mechanism).
--->
<chain:throw
type="File.InvalidExtension"
message="File extension was not valid"
detail="The file you uploaded was not valid. Only CSV files can are allowed."
/>
</cfif>
<!--- Catch any errors that were thrown. --->
<chain:catch>
<!--- Add error message. --->
<cfset ArrayAppend(
arrErrors,
"There was a problem uploading the file."
) />
</chain:catch>
</chain:try>
<!---
For the second step, we are going to process the
uploaded file into a query.
--->
<chain:try>
<!--- Process uploaded document. --->
<cfset qData = ProcessUpload(
"#REQUEST.UploadDirectory##CFFILE.ServerFile#"
) />
<!--- Catch any errors that were thrown. --->
<chain:catch>
<!--- Add error message. --->
<cfset ArrayAppend(
arrErrors,
"There was a problem importing the data file."
) />
</chain:catch>
</chain:try>
<!---
For the last step, we are going to move the query
data into the database.
--->
<chain:try>
<!--- Loop over query and address each record. --->
<cfloop query="qData">
<cfquery name="qInsert" datasource="test">
INSERT INTO [table]
(
[value]
) VALUES (
#qData.value#
);
</cfquery>
</cfloop>
<!--- Catch any errors that were thrown. --->
<chain:catch>
<!--- Add error message. --->
<cfset ArrayAppend(
arrErrors,
"There was a problem inserting the data. Perhaps some of your values were not valid."
) />
</chain:catch>
</chain:try>
<!---
Here, we can process any errors that occurred.
This tag will only be executed if one of the
above tags failed.
--->
<chain:catch>
<!--- Log error. --->
<cfset LogError( CFCATCH ) />
</chain:catch>
</chain:steps>
As you can see here, each step of this algorithm is dependent on the success of the previous step. Each of the chained logical algorithm is called TRY. This is because I am trying to embody what it is doing and why these are all chained - they are trying to execute a portion of the algorithm. And, much like the CFTRY tag, this also has a nested CATCH tag that would catch any errors. The catch here is that if any of the CATCH tags fire (due to an error), none of the subsequent TRY tags would execute. I thought maybe, if we needed a way around this, I could come up with some sort of "ignore" flag like:
<chain:catch continue="true">
... which would say, "An error occurred, but this wasn't an essential step, so let's keep executing the algorithm anyway." Then, at the end of the chained steps, there is a final CATCH tag that would execute if any of the steps threw an error. I guess this would be a bit like the FINALLY tag that I see people mention from time to time... well, not the same intent, exactly.
The biggest problem with a ColdFusion custom tag implementation of something like this is that I wouldn't know how to create the Catching functionality from the parent tag. The conditional execution would not be a problem - that would just be managed through some internal variable.
Does anyone know how to, from a parent tag, create a TRY/CATCH situation around the nested tags of that tag? This is the largest logical hurdle.
Am I crazy? Does this seem like a good idea? Or way more trouble than it's worth? I am not finalized on the names of the tags - this was just to get feedback.
Want to use code from this post? Check out the license.
Reader Comments
I think I'm following you here, and you've shown similar usage of this logic in other examples. For example, in your post on July 17, 2007 regarding upload and email file using ColdFusion, you perform similar usage simply by using <cfif NOT ArrayLen( arrErrors )> to wrap around each "Step" of the process. Doing so appears to keep each progressive step from firing due to a previous error.
Or were you looking to come up with something more akin to <cftransaction>?
Or maybe I'm not following you after all?
This is the first thing that came to mind
http://www.netobjectivesrepository.com/TheChainOfResponsibilityPattern
@Ben:
I've always wished there was a way to throw an error from a CustomTag that would point to the calling template--not the custom tag. I've tried various ways to accomplish this, but the error is always thrown at the tag level.
Also, you'll probably want to add some kind of name attribute to each step and an "if fail, go to." There are often times when you don't want to break the processing chain, but just skip to a different step. I think you'd want to use name attributes as the "goto", that way if order of the steps change, you don't have to change your goto statements.
Actually, after re-reading your post, I caught your mention of NOT liking the IF statements. Sorry. It's just as I was going through your code, I forgot about your comment, and started wondering why you didn't just use an IF statement. Funny.
For what it's worth, I don't mind the IF statements.
I'm not sure if this is what you're looking for, but I've done something similar before. Kinda psuedo-code from your example.
<cftry>
<cfset arrErrors = [] />
<cftry> <!--- step1 --->
<cfset a = 1 / 0 /> <!--- yay, error --->
<cfcatch>
<cfset ArrayAppend(arrErrors,"There was a problem uploading the file.") />
<cfthrow /> <!--- If it's not a major error and you can continue, don't throw --->
</cfcatch>
</cftry>
<cfcatch>
<cfdump var="#arrErrors#">
</cfcatch>
</cftry>
I don't know man, it feels to me this is even more weird to chain or nest catch/try statements on each individual step. If I saw something like this where each subsequent step was in an if/then statement, I would have rewrote it by taking out the if statements and put the whole algorithm in a try/catch block and then make sure each step of the algorithm threw the error on failure. Then write a smart catch block to handle the error or send it back to the caller.
CooLJJ
@Gerald,
That looks interesting, but I think creating classes for each step is going to be way overkill for most applications. But, cool design pattern.
@Bradley,
That's a cool idea. I have never though about throwing errors. I have generally had the "fear" struck in my heart about using exceptions to control logic. But, that could be a nice, clean solution for this.
@CooLJJ,
I tend to agree with you, after 1) having written my post and 2) having seen what Bradley recommended.
@Dan,
That doesn't give me hope :) I'd like to believe that it is possible because errors "bubble" up, right? And I have to feel that in the way of that rising bubble is the parent tag. But, what construct is there to catch errors? It's not like you wrap the TRY/CATCH across the Start / End modes of a tag (it won't parse properly).
Hmmmm. Maybe Bradley's suggestion is the best one.
How about splitting the steps into functions that each return Bool and also write to a shared error catcher? You'd still be using IF statements, but feels cleaner, somehow.
<cfcomponent>
<cfset variables.errorArray = arrayNew(1)>
<cffunction name="mainDealio" returntype="boolean">
<cfif uploadExcel(args)>
<cfif convertExcel(args)>
<cfif saveData(args)>
<cfreturn true>
</cfif>
</cfif>
</cfif>
<!--- if you got here, something failed along the way --->
<cfdump var="#variables.errorArray#">
<cfreturn false>
</cffunction>
<cffunction name="uploadExcel" returntype="boolean">
<cftry>
<!--- upload blah --->
<cfcatch>
<cfset arrayAppend(variables.errorArray, cfcatch)>
<cfreturn false>
</cfcatch>
</cftry>
<cfreturn true>
</cffunction>
<cffunction name="convertExcel" returntype="boolean">
<cftry>
<!--- POIUtil blah --->
<cfcatch>
<cfset arrayAppend(variables.errorArray, cfcatch)>
<cfreturn false>
</cfcatch>
</cftry>
<cfreturn true>
</cffunction>
<cffunction name="saveData" returntype="boolean">
<cftry>
<!--- database blah --->
<cfcatch>
<cfset arrayAppend(variables.errorArray, cfcatch)>
<cfreturn false>
</cfcatch>
</cftry>
<cfreturn true>
</cffunction>
</cfcomponent>
@Jason,
My only problem with that is that I still code very much procedurally and the idea of having to have my form processing logic spread across multiple files is a bit foreign - not to say that it is bad or wrong in anyway, just to say that my perspective might not be the best.
@Ben,
I definitely understand that point of view. My work still tends to split between older Fusebox style and Model-Glue, so I "get" the tension there between procedural and OOP. I guess my suggestion there, though, was simply to use a single CFC, not multiple files. I often find that even when I'm tackling a quick project procedurally, I can still find it useful to drop all related actions into a CFC instead of one or more CFM files.
In this particular example, for instance, all the actions necessary to step through my process would be in one file together, but each step pulled out into CFFUNCTION calls merely to make the flow more obvious and to more easily abstract the error collector. It is, of course, totally possible to do this with CFM files, too, but in a CFC I can see it all in one place.
Just my humble opinion ;)
@Jason,
True true. I do like that it was in one place, very cohesive. Still finding my footing on this issue.
I think you can already do this with try/catch as it is.
<cftry>
<cfset step = 1>
<!--- Do something --->
<cfset step = 2>
<!--- Do something --->
<cfset step = 3>
<!--- Do something --->
<cfcatch>
<cfswitch expression="#step#">
<!--- All the error handling in one place --->
</cfswitch>
</cfcatch>
</cftry>
Steps in order and together, error handling all in one place. Only "extra" code is a single cfset per step and the cfswitch statement, which I think is less code and more readable than many chain:try chain:catches.
@Brian,
Another very good and interesting suggestion. I would have totally never thought about doing it that way. Thanks.
There's a definite school of thought that says you shouldn't use exceptions for flow control. You can google "exception handling" and get more than you'd ever want to read, so I won't say more about that here.
Regarding your dislike of wrapping everything in IF - I agree, but I think Bradley's solution fixes that. The only changes I would make would be just to use AND rather than nested ifs:
<cfif uploadExcel(args)>
and convertExcel(args)
and saveData(args)>
<cfreturn true>
</cfif>
and also reduce the reliance on exception handling in the step functions.
I was going to mention making each step a function on its own which returned true or false, and putting them in a CFIF since it will get short-circuited the first time one of the functions returns false (as Jaime mentioned above)
Glad you're thinking and writing about things like this. Keep it up! =)
@Sammy,
As sad as this might sound, the idea of getting to use short-circuiting logic is almost enough appeal for me to use functions :) I just love short circuiting. Not sure why exactly; I think it makes me really cleaver :)
@Bradley,
I gave your exception-driven methodology a try:
www.bennadel.com/index.cfm?dax=blog:1195.view
I have to say that I really liked it. Thanks for the suggestion.
100 Begin describe what the algorithm does
101 / Perform step 1
102 / If no errors, perform step 2.
103 / If no errors, perform step 3
104 / If no errors, perform step N
105 / If no errors 'SUCCESS'
106 / Else display 'ERRORS'
107 End describe what the algorithm does
108 Set error='no'; Set step=0;
109 Do while error='no'
110 / This loop can only execute if error = 'no'
111 Step=step+1
112 Casentry Select a step
113 Case step is 1
114 Perform this step
115 If error then set error='yes'
116 Case step is 2
117 Perform this step
118 If error then set error='yes'
119 Case step is 3
120 Perform this step
121 If error then set error='yes'
122 Endcase Select a step
123 Enddo while error='no'
124 If error='no'
125 Display 'Success'
126 Else error='no'
127 / You're here if error='yes'
128 Display 'Errors'
129 Endif error='no'
130 Exit
131 End_of_pseudocode
PSEUDOCODE OF C:\KEDITW\SPOK\ALG.TXT 19 APR 2008 11:14:34