- Industry
- Procure AI
- Department
- Engineering
- Team
- Platform and Reliability
- Kind
- Assistant
- Approver
- On-Call Lead
- Code
- ENG-14
It watches the platform's uptime, latency and error rates against each service's objectives, including the business signals clients feel: requisition submission, approval notification delay, invoice ingestion lag from the e-invoicing network, match queue age and payment-file delivery. It tracks error-budget burn, tunes alerts to page only on what clients feel, writes the reliability report and sends anything that silences a paging alert for approval.
How it works
6 steps, and a person signs off before step 6.
- 01Looks it up
Finds the record the request is about in the monitoring system: the service-level indicators, invoice ingestion and match queue metrics, alert history and on-call pages for the services and window named..
- 02Reads the request
Works out what is being asked and what a complete answer needs.
- 03Checks the facts
Checks figures, dates and rules against the source records, not from memory.
- 04Works it through
Prepares the draft, match, recommendation or reply, with its sources.
- 05Asks a person
Hands the work to On-Call Lead with its reasoning before anything goes out.
- 06Acts
Once approved, it acts in the monitoring system: it creates or updates the alert rules and dashboards and posts the reliability report. and records what was done.
What it works with
Your knowledge, your systems, your people.
Knows
The service catalogue and reliability policy: each service's objectives and owners, client contract availability commitments, payment-run and month-end calendars, the alerting standard, and planned maintenance windows.
Reads from
the monitoring system: the service-level indicators, invoice ingestion and match queue metrics, alert history and on-call pages for the services and window named.
Acts in
the monitoring system: it creates or updates the alert rules and dashboards and posts the reliability report.
Hands back
A list with one entry per service, each with service, availability_percent, latency_p95_ms, error_rate_percent, business_signal (an object with name and value, or null), error_budget_left_percent, status (healthy, at_risk or breached), alerts_to_change (a list, each with alert, change and reason) and note.
Asks a person when
The change silences, deletes or raises the threshold of a paging alert, changes a service's objective, or a service has used more than half its error budget.
Settings
Your limits, set by your managers.
| Setting | Default | What it controls |
|---|---|---|
| Default availability objective (%) | 99.9 | Used for a service whose objective is not in the catalogue. |
| Default latency objective (p95, ms) | 500 | Used for a service whose latency objective is not in the catalogue. |
| Invoice ingestion lag objective (minutes) | 30 | Invoices received from the e-invoicing network should be in the match queue within this long. |
| Burn rate that pages | 14.4 | A burn rate at or above this over one hour pages on-call; lower burn opens a ticket. |
| Firings a week that make an alert noisy | 5 | An alert firing more often with no action is proposed for tuning. |
Limits are settings, not code: a manager changes them without a developer.
Rules it always follows
- Measure each service against its catalogue objectives, falling back to 99.9 percent and 500 ms p95.
- Track invoice ingestion lag against 30 minutes, and treat any lag within 24 hours of a client's payment run as paging.
- Page only on symptoms clients feel; a burn rate at or above 14.4 pages on-call, slower burn opens a ticket.
- Propose tuning any alert firing more than 5 times a week with no action, but never silence payment-file delivery or approval-notification alerts.
- Exclude planned maintenance windows from availability and say so; report availability per client contract where it differs from the default.
- Every alert names its owner, runbook and the objective it protects.
See the System Monitoring Agent agent on your own work.
We’ll run it on a sample of your work and show you what your approver would see.