ACME HQDemo work items and pages, made up for Emargen's demo. Not a real company.
Operations / Operations / Operations runbook / Backups

Restore drills

By Mira Holm · created 20 Mar 2023, 14:30 UTC · last updated 28 Sep 2026, 08:45 UTC · version 10

backupsrestoreoperationsiso-27001

Every quarter we restore a backup for real, time it, and write down what went wrong. A backup nobody has restored is a hope, not a backup.

How a drill runs

  1. Open an OPS ticket (Task) a week before. Pick the region: we rotate EU, UK, US, and the EU at least twice a year because most clients live there.
  2. Restore last night's dump to a new instance in the same region.
  3. Start a copy of identity-service against it, in an isolated network with no mail or SMS.
  4. Check: row counts per table against the manifest, the newest updated_at, ten random people with their roles and depots, and a password sign-in with a test account.
  5. Time it from "start restore" to "sign-in works".
  6. Delete the instance. Write the result below and in the ticket.

Log

DateRegionTicketTime to sign-inResultNotes
20 Mar 2023EU–2 h 50 minpassedrestore ran on one thread; too slow
18 Sep 2023EU–1 h 35 minpassedparallel restore
11 Mar 2024EU–1 h 10 minpassedfirst one-client restore too
16 Sep 2024EU–58 minpassed
17 Mar 2025UK–34 minpassedfirst UK drill after the move
16 Jun 2025EU–52 minpassed with a fixKMS permission missing for the drill role
15 Sep 2025UK–31 minpassed
8 Dec 2025EU–47 minpassed
19 Jan 2026US–23 minpassedfirst US drill, region one week old
13 Apr 2026EU–49 minpassed
8 Jul 2026UKOPS-3538 minpassedOskar's first; checklist rewritten
25 Sep 2026EUOPS-4541 minpassedfrom the Frankfurt copy; Mira watched for the ISO evidence

Next: US, in December.

What we keep for auditors

The OPS ticket, the timings, the check results and who ran it. Maja collects them for ISO 27001 (ISO 27001 notes).

Change history

VersionDateWhoWhat
813 Apr 2026MiraQ2 drill
98 Jul 2026OskarOPS-35; drill checklist
1028 Sep 2026OskarOPS-45