Postmortems Nobody Dreads Writing

The worst postmortem I ever read named the engineer who made the mistake in the first sentence. It had no timeline, no action items, and a root cause of "human error." I remember thinking: this document cost an hour of everyone's time and changed absolutely nothing. The same class of incident happened again six weeks later.

Good postmortems are boring documents with sharp edges. They're not tribunals, and they're not résumés either. Here's the format I've used since, adapted from the usual suspects (Google's, Etsy's, incident.io's) and trimmed for teams that don't have a dedicated incident commander.

The structure that works

1. Summary — five sentences, no jargon. What broke, who was affected, how long, current status, one-line root cause. If leadership only reads this section, they should leave with an accurate picture.

2. Impact — in user terms. "8% of checkout requests returned 502 for 23 minutes" beats "ECS task health check failures." Both go in; user terms come first. Dollars, signups, support tickets — whatever your business counts.

3. Timeline — timestamps and facts, in UTC. Every entry is an action or observation, attributed to a role, not a blame:

14:02  Alert: checkout 5xx rate > 2% fires
14:04  On-call confirms elevated errors, starts investigating
14:11  Deploy suspected — rollback initiated
14:18  Errors return to baseline; monitoring for 15 minutes
14:33  Incident declared resolved

The timeline is the heart of the document. It forces you to notice things like "we were blind for nine minutes" or "the rollback took longer than detection" — the real engineering findings.

4. Root cause and contributing factors. Resist "human error." Humans don't fail in isolation; systems let their failures through. The engineer who deployed a bad config did what the system allowed. Ask instead: why did the config pass review? Why didn't staging catch it? Why did production blast so wide before anyone noticed? Three or four contributing factors is normal; a single root cause is often a story we tell to close the ticket.

5. What went well. Genuinely. "Alert fired in 40 seconds," "runbook had the exact command," "rollback was one click." You're marking what to keep funding — teams only ever budget for what broke, never for what saved you.

6. Action items — owned, dated, prioritized. This is where most postmortems die. "Improve monitoring" is not an action item. Try:

• Add canary on checkout path — owner: Priya — due: Sep 30 — priority: high
• Block deploys when config schema validation fails — owner: Sam — due: Oct 7 — high
• Update runbook with rollback steps from this incident — owner: on-call rotation — due: this week — medium

Every item gets a human name and a date or it doesn't exist. Cap it at five. A postmortem with 22 action items and no dates is a wish list.

Running the review meeting

Keep it to 45 minutes, one week after the incident — long enough for facts to settle, short enough that details are still fresh. Roles: facilitator (not the primary responder), scribe, and the people who were in the incident. Everyone else can read the doc.

Ground rules I say out loud every time: we're reviewing the system, not the person; counterfactuals are welcome ("what would have to be true for this not to happen?"); disagreement gets captured, not smoothed over.

The facilitator's main job is catching blame-language in real time and translating it. "John pushed bad code" becomes "what in our path let this change reach production unvalidated?" Same fact, different system.

The metrics worth tracking (gently)

Time to detect, time to mitigate, number of action items completed on time. Trend those at the team level — never by individual, never in performance reviews. The moment postmortems feed performance metrics, people start writing fiction.

Why this pays for itself

The second time our checkout class of incident happened — the one from the beginning — detection took 40 seconds, mitigation took four minutes, and the postmortem was nearly a copy-paste with a note that two prior action items had quietly slipped. That slip became the actual finding.

Incidents are a tax every engineering org pays. Postmortems are how you stop paying full price twice.