nanog mailing list archives
Re: Disaster Recovery Process
From: Sean Donelan <sean () donelan com>
Date: Tue, 5 Oct 2021 10:34:28 -0400 (EDT)
On Wed, 6 Oct 2021, Karl Auer wrote:
I'd add one "soft" list item: - in your emergency plan, have one or two people nominated who are VERY high up in the organisation. Their lines need to be open to the decisionmakers in the emergency team(s). Their job is to put the fear of a vengeful god into any idiot who tries to interfere with the recovery process by e.g. demanding status reports at ten-minute intervals.
A good idea I learned was designate separate "executive" conference room and "incident command" conference room.
Executives are only allowed in the executive conference room. Executives are NOT allowed in any NOC/SOC/operations areas. The executive conference room was well stocked with coffee, snacks, TVs, monitors, paper and easels.
An executive was anyone with a CxO, General Counsel, EVP, VP, etc. title. You know who you are :-)
One operations person (i.e. Director of Operations or designee for shift) would brief the executives when they wanted something, and take their suggestions back to the incident room. The Incident Commander was God as far as the incident, with a pre-approved emergency budget authorization.
One compromise, we did allow one lawyer in the incident command conference room, but it was NOT the corporate General Counsel.
Current thread:
- Re: facebook outage, (continued)
- Re: facebook outage Ryan Brooks (Oct 04)
- Re: facebook outage Bill Woodcock (Oct 04)
- Re: facebook outage Patrick W. Gilmore (Oct 04)
- Re: facebook outage John Lee (Oct 04)
- Re: facebook outage Niels Bakker (Oct 05)
- Re: massive facebook outage presently Jay Hennigan (Oct 04)
- Re: massive facebook outage presently Jorge Amodio (Oct 05)
- Disaster Recovery Process Jared Mauch (Oct 05)
- Re: Disaster Recovery Process Karl Auer (Oct 05)
- Re: Disaster Recovery Process Jared Mauch (Oct 05)
- Re: Disaster Recovery Process Sean Donelan (Oct 05)
- Re: Disaster Recovery Process Wolfgang Tremmel (Oct 06)
- Re: Disaster Recovery Process Jeff Shultz (Oct 05)
- Re: Disaster Recovery Process jim deleskie (Oct 05)
- Re: Disaster Recovery Process Niels Bakker (Oct 05)
- Re: Disaster Recovery Process jim deleskie (Oct 05)
- Re: Disaster Recovery Process Jamie Dahl (Oct 05)
- Re: Disaster Recovery Process Warren Kumari (Oct 05)
- RE: massive facebook outage presently Jean St-Laurent via NANOG (Oct 04)
- Re: massive facebook outage presently Bryan Fields (Oct 04)
- Re: massive facebook outage presently Matt Hoppes (Oct 04)
