Page on-call when SMS delivery receipts lag more than five minutes

Project: Engineering
Type: Task
Status: Done Done
Priority: Highest
Reporter: Oskar Lund <oskar.lund@acme-hq.example>
Assignee: Elif Yildiz <elif.yildiz@acme-hq.example>
Labels: incidentphone-codes
Components: notifierinfrastructure
Fix versions: notifier 2026.06.24
Sprint: Sprint 45
Created: 17 Jun 2026, 09:30 UTC
Updated: 24 Jun 2026, 13:00 UTC
Resolved: 24 Jun 2026, 13:00 UTC

Description

OPS-33: Textbridge accepted every code from 07:05 and delivered almost none to UK and Irish numbers. We watched whether Textbridge accepted a code, not whether it was delivered, and heard from a customer 35 minutes in.

Alarm: delivery receipts more than 5 minutes behind, or below 90% of codes sent over 10 minutes, per country route. Pages on-call.
Added to Confluence OPS › Monitoring and alerts and ENG › Runbook: SMS codes not arriving.

Linked issues

relates to ENG-1488 Page on-call when refused or undelivered texts pass 5% in 10 minutes Closed
relates to OPS-33 Textbridge outage: SMS codes to UK and Irish numbers delayed for about two hours Done
relates to ENG-1620 notifier: page when any country's median receipt lag passes 2 minutes over 10 minutes Done

Comments (1)

Elif Yildiz · 24 Jun 2026, 13:00 UTC

Live. Replayed Wednesday morning's numbers against it: it would have paged at 07:11.