Restore first, understand later
The instinct to find the root cause before acting is strong and usually wrong. If a rollback, a restart, or a failover restores service, do that first and investigate from a position of stability.
The exception is data corruption, where acting without understanding can make things permanently worse. Knowing which situation you are in is the first judgement call.
Name a coordinator
One person owns the incident: tracking what has been tried, deciding what happens next, and handling communication. Critically, that person should not also be debugging, because the two roles compete for exactly the same attention.
Without a coordinator, several people investigate the same theory, nobody tracks what has been ruled out, and stakeholders interrupt whoever is closest to a fix.
Communicate on a cadence
Agree an interval and stick to it, even when the update is that there is no update. Silence causes stakeholders to seek information directly from the people working the incident, which slows resolution.
Say what is known, what is not, what is being tried, and when the next update will come. Avoid predicting resolution times early; a missed estimate costs more trust than an honest uncertainty.
Keep a timeline as you go
Record actions and observations with timestamps while they happen. Reconstructing this afterwards is unreliable, and the timeline is the raw material for the review.
A shared channel where every action is stated as it is taken does this almost for free, and it also prevents duplicated work.
Review without blame
The purpose of a review is to find the conditions that allowed the failure, not the person who made the change. Where reviews find people, engineers stop volunteering information and the organisation loses its ability to learn.
Ask what made the mistake easy, what made it hard to detect, and what made it slow to fix. Those three questions produce actions; asking who did it does not.
Close the actions
Reviews generate improvements that frequently go unimplemented, and the same incident recurs. Assign each action an owner and a date, and track them like any other work.
An incident that recurs after a review is a more serious failure than the original, because the organisation had already paid for the lesson.
