A Big Hole In My Understanding Of ColdFusion Scheduled Tasks
Posted May 9, 2011 at 9:34 AM by Ben Nadel
Last week, I was reviewing one of our ColdFusion scheduled tasks with Ryan Jeffords when I suddenly had a moment of insight. In a flash, I realized that for years, I have completely misunderstood the way durations were defined at the "daily" level of a scheduled task. What I finally came to understand in that moment was that scheduled tasks use both a date range and a time range to define their execution window.
If you are not familiar with a ColdFusion scheduled task, setting one up in the ColdFusion administrator looks like this:
| || || |
| || |
| || || |
As you can see, the ColdFusion scheduled task setup allows for a date-based duration and a time-based frequency. In this particular setup, I have the frequency set to run daily, every hour, until the end date (which has been omitted - it will run forever).
Now, before last week, this is how I interpreted this particular ColdFusion scheduled task setup:
"This task will start running on May 9, 2011 at 9:00 AM and will execute every hour thereafter until the end of time."
As it turns out, this is completely wrong. In hindsight, my error seems somewhat silly; but for some reason, the meaning of this scheduled task configuration has never been completely obvious to me. Now, post-revelation, this is how I interpret this particular ColdFusion scheduled task setup:
"This task will start running on May 9, 2011 and will run every day until the end of time. On each valid day of execution, the schedule task will start running at 9:00 AM and will run every hour thereafter until the end of that day - at which point, it will stop until 9:00 AM of the next valid day of execution."
Can you see the critical difference in understanding? In the first interpretation, the start time is linked directly to the start date. In the second interpretation, the date range and the time range will execute more independently (within context).
| || || |
| || |
| || || |
I don't think I've ever caught this problem before because luckily, it's never been a business-critical misunderstanding. After all, the scheduled task will still execute every day. The problem with the first interpretation, however, is that the scheduled task will not be running at all for the first 9-hours of each day. And, depending on your situation, this may or may not ever throw up a red-flag.
This ColdFusion scheduled task misunderstanding is, I have to admit, a little bit embarrassing. I'm putting it out here in case there are others like me, who never quite connected the dots between date ranges and time ranges.
Yeah, I can't tell you how many times I've made a daily task and just entered 12:00AM 11:59PM. I'm _very_ happy tasks are getting updates in the next rev of ColdFusion.
The funniest part of this is that every time I set up a scheduled task, I always thought to myself, "Ugg - why do they separate the date and the time." ... of course, I just assumed *they* made a bad decision - it never occurred to me that I might not be understanding it :)
Actually, I got confirmation from Adobe that Scheduled Tasks should work the way you were expecting them to work, and not the way they do.
The 9 hour gap, as in your scenario, is not valid behaviour.
At least that's what got reported back to me.
As Ray says, they've long overdue an overhaul, and thankfully it's happening.
I have to say, now that I understand *how* they actually work, I'm a bit conflicted. Part of me feels that having a somewhat independent date/time range gives the most amount of flexibility. I'm not so sure that I want them to change.
What might be better is simply to explain it more clearly on the configuration screen?
Yeah, it is very confusing. I can't tell you the number of times people have told me that the "scheduler is broke" when in fact, they just misunderstood what they told it to do.
Glad to see the the scheduler getting some love soon.
I was definitely one of those people :)
Coming from a Cron background this took some getting used to. I've lost count on the number of tasks I configured to start at 00:01 and end at 23:59 just to get half decent repeated tasks working.
To be honest (and this isn't ColdFusion bashing) I wouldn't use the current scheduled task system for anything business critical. The implementation (at least on Windows/JRUN servers) is really quite weak and has locking issues which can result in tasks disappearing.
I'm looking forward to the CF10 changes but until then will continue to use Cron and cURL to do my scheduled task work.
I've definitely had some issues with scheduled tasks in the past, unrelated to this problem. I've had weird SSL issues and situations where it would run fine when you tested it, but wouldn't run properly when scheduled.
I've never used Cron though; I am not even sure where that stuff is set up (though I am sure Google could answer that in a matter of secondsS).
I've had issues with inflexible scheduling on ColdFusion. Scheduling tasks to be performed on every Tues, Wed & Thurs at 3am requires 3 different tasks. In a Windows environment, setting using the NT Task Scheduler was almost as painful. We finally started using nnCron as a replacement:
It uses an editable configuration file, so you could even write a ColdFusion GUI to manage tasks. (Check out the documentation so that you can see what else it's capable of.)
Not run into the SSL stuff but can have certainly had issues with locking the neo-cron.xml file which was enough to make me run away very quickly.
Cron on *Nix is pretty simple and with cURL you just "get the URL" in the same way as the scheduled task manager. Try running "crontab -e" from a terminal on OSX.
As you surmised there's lots of documentation for it around the net (not least of which is the man file which is pretty informative) and cURL is simplicity itself to use (albeit with a massive array of options for authentication, SSL etc which can make it quite daunting if you read the official docs.)
I just hope the Adobe guys do something to make the new Scheduled task manager uber reliable - there's no point in having an unattended task manager if you can't trust it to run unattended :)
I've not personally had to deal with tasks distributed on different days; but, I can see how that would be annoying to manage multiple tasks for the same "intent."
I'll take a look at this other cron stuff. It's all new to me.
I also prefer the CRON & CURL combination because it doesn't consume a CF thread or run the risk of timing out when connecting to external hosts. I can also run it on any computer so that it never impacts the CF server. I've had to add extra success/fail flags & notifications to the individual scripts when they are called by CURL.
I like it - especially the fact that it can be run by any computer; that just sounds powerful :)
I figured this out a while back with a task that was supposed to be updating weather all day. First time, for me, I already knew one of your posts.
I've been a frequent user of scheduled tasks since the 3.x days and the way it works is fine with me. It's very logical once you have that (delayed) mini moment of revelation that Ben did! :-D
It's actually quite flexible and I hope any change in future releases just adds to the functionality and doesn't completely change how it works. I can't actually think how they could improve it anyway, but I look forward to being surprised by someone else's innovation!
Ha ha - awesome :)
I'm with you on this - the issue was not with the functionality of the scheduled task; that's actually pretty cool. The issue was more with the gap between understanding and functionality. Like now that I *get* it, I wouldn't want to change it.
Or, at least make the start/end time optional. If they are left out, the system should assume 12AM - 11:59PM.
Ben you're not alone. Before I read this blog post, I interpreted the same.
Awesome my man - glad to know that we were in the same boat. Glad to also finally have this beast cleared up!
hey, this may be a really dumb question, but do you know if there is any way to set up a scheduled task, via code, if you do not have access to the ColdFusion administrator. Like, for example, to be able to write an applicaiton that would do it? I'm not asking for evil purposes, an example of a circumstance where this would be desired is this: say you worked for a company, and you worked on specific sections of code for them, but your access to ColdFusion administrator was limited. But you had a task that needed to be run on a regular basis.
The answer, I am sure, is probably no, but I thought I would ask.
There is a cfschedule tag. It allows you to setup scheduled tasks.
oh man. That's cool! thanks a heap!!!!
As for the issue of running multiple scheduled tasks, I tend to set up a single scheduled task in the CF Administrator which runs every 5 minutes (ish). When that task runs, the page that's called looks at a table of tasks to determine which, if any, should be run. For example, a "cleanup database" routine might run once at 3AM, but a "send queued email" might run every time. That model has, for the most part, worked nicely for me. Just my .02.
I think that's the winning approach. Plus, that way, you don't need multiple developers to have access to the CF Admin in order to be able to set up new tasks (if that's a security concern). I like that approach as well because you can have cross-cutting concern around the tasks like error handling, logging, etc..
Not sure if this is the best place to ask ... but here it is:
Is there a limit for interval value when setting up a schedule job?
I need to schedule a job every 12 weeks which is 7257600 seconds.
Hmm, to be honest, I've never had a task that didn't run daily or weekly. Every 12 weeks is a tough one to work with. I could be talking poorly here, but for that, it might be best to store the task details in a database and then just run it every day (and use the database to see if the task should truly be executed).
This is actually how I run most of my tasks these days. I have a centralized task that runs like every 5-15 minutes; then, it hits a database to see what actual tasks should be run.
I'll see if I can write that concept up - I don't think I ever blogged about it before.
I've done something like that before as well. So um, +1 Ben. ;)
@raymond. which Ben gets the +1. (-;
Oh sorry - meant Nadel, and the idea of having the task run every day and internal logic then determines if it REALLY needs to run.
Thankfully scheduled tasks are getting Awesome Sauce in Zeus.
You get +1 for having the coolest name in the world ;)
Is there a way to write a "dependency" routine where a task would only executed only if previous scheduled tasks is complete.
Is this possible? Maybe a feature request in new CF?
Fixed time schedules are no good if the previous task did not successful execute.
I would personally keep a record of scheduled tasks in a database, including information about when they should execute, when they last executed, and dependencies. I'd think it would be pretty straightforward to write CF code around that.
How do you know that it's completed? Do I just right a routine to do a check of some sort and if that value is what I want then I go on to the next step?
My scenario is this:
1. script A will read all new records from a certain point in time and inserts it into another table (Table X).
2. Script B will update a value in Table X.
3. Script C will delete all records in Table X.
Take it as a given that I don't know your system, nor what you require, so my advice might be no good. However, I'd suggest something like:
Have script A set up to run at 1AM, B at 2AM and C at 3AM. At the completion of each script, update your "scheduledTasks" table with a timestamp indicating completion. So, script A runs and takes one minute, and so you update the "lastUpdatedDateTime" datetime column on the scheduled tasks table with '2011-08-12 1:01:00'.
Then, when it comes time for script B to run, your routine would first make sure that Script A not only successfully completed, but successfully completed the same day.
Is that helpful at all?
I already have scheduled the 3 tasks. I guess I'm struggling with how to update my table with a timestamp to indicate that it's "done". How do I know it's done? should I try and retrieve a http status code somehow after the page loads and then check to see what it is and if it's okay, then enter the timestamp?
Sorry. How long does it take each scheduled task to execute, approximately? I'm guessing longer than a standard page request?
Depending on the number of records, it actually should not take more than say 3 mins .. actually it only takes about 3 seconds =)
Sorry for the delay. On vacation.
I'd suggest updating a database record immediately before you start the routine to set a status like "task in progress". Then, execute the task. When the task is done, update the status to "complete". If your task takes only three seconds to complete, you should be able to do all of that in one routine.
I don't know if this is documented, but it appears that a scheduled task with no end date is actually limited to about eight weeks in the developer edition. I deduced this from a task that would no longer work until I manually deleted and programatically added it again.
Chris, can you file a bug report for that please?
Bug report has now been submitted.
Very strange! I can't say I've ever really had a scheduled task run for any significant period of time in development. I typically just trigger those beasts manually to make sure they work (and then set up the scheduled tasks in production). Good catch!
You know, it just goes to show you, even with a system that was 100% Perfect, if you have a scheduled task, you should probably have some way of verifying it, or making it 'vocal' so you know it actually run. I don't necessarily mean you need an email for a task running every 5 minutes, but, the slower a task, the easier it may be to forget about it.
A couple of other things...
If manually entering a task (using Administrator), you need to enter the correct format for your locale. However, if programmatically entering a task (using cfschedule), and you enter the wrong format, it will be automatically reformatted.
In addition, the minimum frequency you can enter in Administrator is one (1) minute and one (1) second. However, if using cfschedule, you can enter 60 seconds, or even 30 seconds, and it works, although, Administrator > Debugging & Logging > Scheduled Tasks, incorrectly displays: Daily every 1 min(s)... (Duh).
I commented on the cfschedule page about this on 25 September 2011
I am facing a weird situation here. Whenever I run a scheduled task manually, the last run column do not update the date and still shows the last run as the date when scheduled task ran before manual run.
Is this a config issue?