Running Incident Commander At InVision App Using Angular 4
At InVision App, when something goes wrong in production, we open an "Incident." During the Incident process, one of the team members must act as a liaison between the management / support teams and the engineers that are actively investigating the problem. This point person is known as the "Incident Commander"; and, is responsible for posting regular updates to our team's #Incident Slack channel. If you know me, then you know that I'm fanatical about my code formatting. When I act as the Incident Commander, I feel just as strongly about my Slack updates. As such, I've created a small Angular 4 application to help me codify and transform my updates into strictly formatted Slack messages.
Run the Incident Commander application on GitHub.
View the Incident Commander code in my Incident Commander project on GitHub.
The Incident Commander application has a few nice features:
- Uses localStorage - The current incident data is stored in the browser's localStorage API. This way, if you accidentally close your browser tab; or, if you need to come back to the incident later on; you can simply re-open the Incident Commander application and the most recent incident data is reloaded from localStorage.
- Uses Local Timezone - The #Incident channel updates must be posted using the EST / EDT timezone since the company hours are (more or less) based on the east coast. This is great if you're on the east coast; but, horrible for everyone else. The Incident Commander application allows you to input messages using your local timezone; then, the application automatically converts your local time to EST / EDT time (as best it can) when calculating the Slack message.
- Configurable Formatting - Since incident investigations can get quite complex, the Incident Commander application allows the Slack message formatting to be changed over time. This way, when you're mid-incident, you can keep the updates short and compact. Then, when the incident is over (or the day is done), you can post a more comprehensive, more readable version of the investigatory timeline.
The Angular 4 application uses a simple form interface, hosted on GitHub pages:
... and produces Slack messages that look like this:
This has real-world value for me personally; though, it could be broadly applicable to anyone that needs to manage an incident at work. If nothing else, this was a fun adventure in Angular 4. It's been a little while since I've worked with Angular (after having started to dive more deeply into Node.js recently). So, this was a nice way to get back in the habit of client-side JavaScript development.
Reader Comments
This is awesome Ben, the "Incident Commander" role is something which is talked about a lot in the customer service world for managing public comms during downtime.
Someone who sits between the customer service team and the engineering gang making sure both groups are singing from the same hymn sheet when talking with customers.
They discuss it on the Support Breakfast podcast (along with a bunch of other crisis management tips) - https://supportbreakfast.com/2017/01/29/support-breakfast-podcast-episode-8-sending-out-an-sos/
I think it's a role you see in lots of industries, think about the Maître' D' in a restaurant, who handles things between front-of-house and the chefs - critical role in busy stressful situations.
@Rob,
It's been great for us. One of the biggest benefit is that it doesn't overload the engineers into having to actively *debug* the problem *and report* on it. They can totally focus on the primary objective and the IC can worry about letting the right people know the stuff.
I've actually started to really enjoy being IC - I feel like the puppet master, though I'm not really doing any of the work :D
@All,
I added crude import / export feature until I can figure out a more global persistence mechanism:
www.bennadel.com/blog/3313-adding-import-export-functionality-to-the-incident-commander.htm
Right now, I'm sticking with localStorage because an Incident may contain sensitive company information; and, I don't want to just store that willy-nilly in some shared space.