Anatomy of an Incident by Unknown

Author:Unknown
Language: eng
Format: epub
Published: 2022-01-20T21:16:37+00:00

Figure 4-3. Outage lifecycle: time to repair

Train the responders

Unprepared on-callers lead to longer repair times. Consider conducting on-call training sessions on topics such as disaster recovery testing, or running the Wheel of Misfortune exercise we mentioned earlier. Another technique is a mentored ramp-up to on-call. Having on-callers work in pairs (“pair on call”), or having an apprenticeship where the mentee joins an experienced on-caller during their shifts (“shadowing”), can be helpful in growing confidence in new teammates. Remember that on-call can be stressful. Having clear incident management processes can reduce that stress as it eliminates any ambiguity and clarifies the actions that are needed. 5

Establish an organized incident response procedure

There are some common problems regarding incident management.

For example, lack of accountability, poor communications, missing hierarchy, and freelancing/heroes can result in longer resolution times, add additional stress for the on-callers and responders, and impact your customers. To address this, we recommend organizing a response by establishing a hierarchical structure with clear roles, tasks, and communication channels. This helps maintain a clear line of command and designates clearly defined roles.

At Google, we use IMAG (Incident Management at Google), a flexi-ble framework based on the Incident Command System (ICS) used by firefighters and medics. IMAG teaches you how to organize an 5 See Jesus Climent’s post “Shrinking the Time to Mitigate Production Incidents—CRE

Life Lessons”, December 5, 2019.

Reducing the Impact of Incidents | 31

Download

Copyright Disclaimer:
This site does not store any files on its server. We only index and link to content provided by other sites. Please contact the content providers to delete copyright contents if any and email us, we'll remove relevant links or contents immediately.