- Industry
- Retail
- Department
- Engineering
- Team
- Platform and Reliability
- Kind
- Assistant
- Approver
- Head of Platform
- Code
- ENG-14
It watches the uptime, latency and error rates of the online store, app, checkout, payment callbacks, store tills and the order and stock feeds, and raises alerts on the business signals too, such as orders per minute falling or tills going offline. It writes the alert with what is affected and who owns it.
How it works
6 steps, and a person signs off before step 6.
- 01Looks it up
Finds the record the request is about in the monitoring service: the metrics, logs and business signals for the services and stores named, with baselines..
- 02Reads the request
Works out what is being asked and what a complete answer needs.
- 03Checks the facts
Checks figures, dates and rules against the source records, not from memory.
- 04Works it through
Prepares the draft, match, recommendation or reply, with its sources.
- 05Asks a person
Hands the work to Head of Platform with its reasoning before anything goes out.
- 06Acts
Once approved, it acts in the on-call alerting service: it raises or updates the alert and pages the owner. and records what was done.
What it works with
Your knowledge, your systems, your people.
Knows
The monitoring guide: the service level objectives per service, the business signal baselines (orders per minute by hour and day), the alert routing and on-call rota, and the store till health checks.
Reads from
the monitoring service: the metrics, logs and business signals for the services and stores named, with baselines.
Acts in
the on-call alerting service: it raises or updates the alert and pages the owner.
Hands back
A list with one entry per alert, each with service, signal, value, baseline, severity, affected (shoppers, stores or channels), owner and suggested_first_step.
Asks a person when
It proposes paging leadership or switching the site to a holding page.
Settings
Your limits, set by your managers.
| Setting | Default | What it controls |
|---|---|---|
| Checkout error rate that alerts (%) | 2 | A checkout error rate above this pages the on-call engineer. |
| Order rate drop that alerts (%) | 30 | Orders per minute this far below the baseline for the hour alerts. |
| Offline tills in one store that alert | 2 | This many tills offline in one store alerts store support. |
| Stock feed lag that alerts (minutes) | 15 | A stock feed older than this alerts, to avoid overselling. |
Limits are settings, not code: a manager changes them without a developer.
Rules it always follows
- Page on-call when the checkout error rate passes 2 percent or orders per minute fall 30 percent below the baseline for that hour and day.
- Alert store support when 2 or more tills are offline in a store during trading hours.
- Alert the integration owner when a stock feed lags more than 15 minutes.
- Group alerts from the same cause into one, and name the owner and the shoppers or stores affected.
- Never alert on a single data point; confirm over at least three readings.
See the System Monitoring Agent agent on your own work.
We’ll run it on a sample of your work and show you what your approver would see.