I remember the moment that I got my first pair glasses down at Union Square in New York City. I was in my early 20s. It was shocking. Suddenly, the world was sharp. Street signs were visible. Presentations were visible. Nothing was fuzzy. The world was clear. This is kind of what it felt like when I started using FusionReactor to monitor the performance of my ColdFusion servers. That which was a blackbox was suddenly a fountain of information, offering up all kinds of ways in which I could improve the performance of my code and the user experience (UX) of my clients.
I had heard of FusionReactor in the past; but, it wasn't until servers started dying at InVision that I actually installed it, primarily at the behest of Charlie Arehart. While it was emotionally hard to wrap my head around a new piece of software while my own software seemed to be falling apart, FusionReactor quickly proved to be invaluable. For the first time, I could see actual metrics about my servers and the InVision software:
- How many requests were coming in.
- How long requests were taking to process.
- How many JDBC requests were being made.
- How long JDBC requests were taking.
- Which requests were making which JDBC requests.
- Which JDBC requests were taking the longest.
- Which JDBC requests were slow.
- How many CFThreads were running.
- How many CFThreads were being queued up.
- What CFThreads were doing.
- How performance was changing over time.
- Which requests were returning non-2xx status codes.
- How hot the CPU was running.
- Where code was running at this very moment... right... now.
And that's when I was actively looking at the FusionReactor console. When I was doing something silly, like sleeping, FusionReactor was configured to send me stack-traces anytime my server passed a threshold of running requests. So, for example, if there were suddenly 130 concurrently-running requests at 3AM, FusionReactor would dump the entire stack-trace of the ColdFusion instance and email it to me for review. This is what lead to my GitHub-based FusionReactor protection alert parser (which I think is broken at the moment).
Now, FusionReactor is pretty great already; but, at dev.Objective() 2015, David Tattersall, CEO of Intergral, showed me stuff coming up in the next release of the software. It was kind of mind-blowing. They are giving FusionReactor the ability to be a fully-interactive request debugger. I'm talking about pausing running requests, stepping through calls, viewing variables, live-editing code, and resuming. Holy chickens! I can't wait.
And the truth is, I really know and use a fraction of what FusionReactor has to offer (like it's integration with FusionAnalytics for big data performance metrics). One of these days, I have to sit down and really learn the tooling end-to-end.
It's gotten to the point where, if I don't have FusionReactor installed on a server, I panic. I feel like I have no idea what's going. It's like someone ripped away my security blanket. It's definitely a critical component of what has made InVision App, Inc. so successful.
Great to read this, Ben. Thanks for the kind regards and of course most important great to see you encouraging others to consider FR. Like you, I feel that a server without it is challenging to really understand.
And beyond the alerts, let's also praise the logs, which can help with post-mortem analysis. Its logs are like having the ekg machine to see the vital statistic.
And all this is for CF, Lucee, Railo, BlueDragon, or whatever JEE serve or java app one may need to monitor/support.
And another thing shared at the conference was the upcoming "FR in the cloud" version, which will hold the monitoring data and serve the monitoring interface on remote servers that the FR folks will manage for us (all private to each customer). This will be very familiar to those aware of NewRelic. And given that FR has unique and direct insight into CF/CFML engines, it definitely adds much that NR does not offer. Heck, it offers lots that the CF Server Monitor (and SeeFusion) don't offer.
It really is the best server monitor for CFML engines, and it's only getting better all the time.
Just one point of clarification about the new "debugger" you mentioned. Some readers may think, "well they've always offered FusionDebug", but this new tool (called for now "unattended production debugging') is a totally new aproach.
Where other debuggers run from within an IDE (like Eclipse or CF Builder), this UDP feature runs entirely within the server, thus the "unattended aspect in its name.
There will surely be more news on it to come, whether from the FR folks, or you or me or others in the community. It is indeed a revolutionary new tool. In fact, they have a patent pending on it!
Keep up the great work on your blog and with Invision. Wishing you,Clark, and the rest of the gang all the best.
My pleasure, fine sir.
I wish the log files were something that I was more comfortable with. Unfortunately, I never really took the time to learn too much about them. I think that's in part because I don't know much about log-reading tools and, in part, because I just didn't have a good mental model for what I was looking for. I think usually, when we did look at the log files, we were looking for the number of threads that were queued, which I don't think we can see even from the full stack-trace (since I think it only shows the state of running threads).
As far as Fusion Debug, I am not sure that I have ever heard of it :D And, on the topic of things I don't know much about, I would have also loved to learn more (or anything) about Fusion Analytics. As much as FR is great for the current-state monitoring and the slow queries, I think (or believe) that Fusion Analytics would have provided more analysis over time to better understand problems in a broader context.