When a serious technical incident ends, everyone wants to move on.
The service is back. Customers can log in again. The urgent messages have stopped. The exhausted team would rather close the ticket and return to normal work.
That is exactly when one of the most valuable parts of incident response should begin: the post-mortem.
Done badly, a post-mortem becomes a bureaucratic report or a disguised search for somebody to blame.
Done well, it turns failure into organisational learning.
Start with a timeline
Memory changes quickly after a stressful incident.
Build a factual timeline while evidence is still available.
When did the first failure occur? When did monitoring detect it? When did somebody begin investigating? What actions were taken? Which action restored service? When were users informed?
Use logs, deployment records, alerts and messages rather than relying only on recollection.
A timeline often exposes important gaps. Perhaps the system failed at 09:05 but nobody knew until a customer reported it at 09:37. That 32-minute gap may be more important than the individual code defect.
Separate cause from blame
A useful post-mortem asks how the system made the failure possible.
Suppose an engineer deployed a configuration change that caused an outage.
“The engineer made a mistake” is technically an explanation, but it teaches almost nothing.
Why could one configuration change disable the whole service? Why was there no validation? Why did staging not expose the problem? Why was rollback slow? Why did monitoring fail to detect the impact immediately?
Human error is usually the beginning of analysis, not the end.
Blameless does not mean responsibility-free
The phrase blameless post-mortem can be misunderstood.
It does not mean pretending nobody made a mistake.
It means creating enough psychological safety for people to describe what they actually did and why it seemed reasonable at the time.
If engineers expect punishment for honest disclosure, the organisation receives a sanitised version of events. That makes recurrence more likely.
Accountability asks people to learn and improve.
Blame encourages people to protect themselves.
Ask what made sense at the time
Hindsight makes failures look obvious.
After an incident, everyone knows which alert mattered and which deployment was dangerous.
The people involved did not possess that knowledge beforehand.
A good review reconstructs the information available at each decision point.
Why did the engineer believe the change was safe? Why did the responder investigate the database first? Why was an alert dismissed?
Understanding the local logic of a decision is essential if we want to improve future decisions.
Find contributing factors
Complex incidents rarely have one cause.
A useful post-mortem may identify:
- a software defect;
- inadequate test coverage;
- an undocumented dependency;
- excessive permissions;
- missing monitoring;
- a confusing interface;
- an unsafe deployment process;
- staffing or communication problems.
The goal is not to produce the longest list possible. It is to understand which conditions increased the likelihood or impact of the failure.
Actions must be concrete
“Improve testing” is not an action item.
“Add an integration test covering failed payment-provider responses before the next release” is.
Good actions have an owner, a scope and a realistic priority.
Not every observation requires a project. Teams can drown in post-mortem actions that are never completed.
Prioritise changes that reduce the probability of recurrence, reduce the blast radius, improve detection or shorten recovery.
Include what went well
Incident reviews often focus entirely on weakness.
That misses valuable information.
Perhaps an engineer noticed an unusual metric quickly. Perhaps the rollback process worked perfectly. Perhaps clear communication prevented duplicate work. Perhaps a recent backup made recovery possible.
Those strengths should be preserved.
Resilience grows by reinforcing what worked as well as correcting what did not.
Share the learning
A post-mortem hidden in a folder nobody reads has limited value.
Make reports searchable. Discuss important incidents with relevant teams. Link recurring themes. If several outages involve the same dependency or deployment weakness, the organisation needs to see the pattern.
Over time, post-mortems become a form of institutional memory.
New engineers can learn from incidents they never experienced.
The real measure of a post-mortem
A polished document is not the goal.
The question is whether the system is better afterwards.
Did detection improve? Is recovery faster? Is the dangerous operation harder to perform accidentally? Do people understand the dependency more clearly? Has the runbook been updated?
If nothing changes, the organisation has documented failure rather than learned from it.
Failure is expensive.
A good post-mortem makes sure we do not pay the same tuition twice.
