Skip to main content
Ben Nadel at NCDevCon 2011 (Raleigh, NC) with: Hemant Khandelwal
Ben Nadel at NCDevCon 2011 (Raleigh, NC) with: Hemant Khandelwal ( @khandelwalh )

Working Code Podcast - Episode 174: When Good Software Goes Bad

By on
Tags:

On today's show, we talk about incidents and outages at work. Incidents are a fact of life. If you depend on a file system or a database or a third party vendor, at some point, something will break and your service will be degraded. Customers freak out (rightly so); and, it becomes a cross-team effort to try and find the problem, fix it, and effectively communicate updates back to your customers. There's no right way to do this. But, one could argue that there are definitely wrong ways to do this.

At work, we use a (free) tool that I created called Incident Commander (using Angular and Firebase). It creates a centralized place in which to record key events within the incident remediation timeline. This timeline can then be automatically copied as a Slack message, which can then be used to communicate updates more broadly.

All that and more on this week's show:

... featuring these beautiful, beautiful people:

With audio editing and engineering by ZCross Media.

For the full show notes and links, visit the episode page. And, be sure to follow the show and come chat with us on Discord! Our website is workingcode.dev and we're @WorkingCodePod on Twitter and Instagram. New episodes drop weekly on Wednesday.

Push button meme with two buttons: one for paging the on-call SRE and one for doing nothing (and watching it all burn). This is how Ben is at 2AM.

The above was generated from the Two Button Meme Generator.

Reader Comments

Post A Comment — I'd Love To Hear From You!

Post a Comment

I believe in love. I believe in compassion. I believe in human rights. I believe that we can afford to give more of these gifts to the world around us because it costs us nothing to be decent and kind and understanding. And, I want you to know that when you land on this site, you are accepted for who you are, no matter how you identify, what truths you live, or whatever kind of goofy shit makes you feel alive! Rock on with your bad self!
Ben Nadel