Incident Response Agent
Diagnoses outages and drafts incident reports.
- Industry
- Oil & Gas
- Department
- Engineering
- Team
- Platform and Reliability
- Kind
- Investigator
- Approver
- Incident Commander
- Code
- ENG-15
It diagnoses outages of the company's digital services, such as the permit-to-work system, the historian replica, the trading platform's feeds and the fuel-card authorisation service, from alerts, logs and recent changes, ranks the likely causes and proposes the safest mitigation. It drafts the incident report and tells site operations at once when an outage affects permits or field work. **The problem it solves:** Why the service failed: the alerts, logs and recent changes, the dependencies involved and whether site work relying on it is affected.
How it works
6 steps, and a person signs off before step 6.
- 01Looks it up
Finds the record the request is about in the monitoring system: alerts, logs and changes..
- 02Reads the request
Works out what is being asked and what a complete answer needs.
- 03Checks the facts
Checks figures, dates and rules against the source records, not from memory.
- 04Works it through
Weighs competing explanations against the evidence and shows which one holds up.
- 05Asks a person
Hands the work to Incident Commander with its reasoning before anything goes out.
- 06Acts
Once approved, it acts in the incident tool: it updates the incident with the diagnosis and report draft. and records what was done.
What it works with
Your knowledge, your systems, your people.
Knows
The incident runbooks: service dependencies, severity levels, the site notification procedure when permit to work or emergency notification is down, and the postmortem template.
Reads from
the monitoring system: alerts, logs and changes.
Acts in
the incident tool: it updates the incident with the diagnosis and report draft.
Hands back
An object with causes (ranked, with evidence), mitigation, site_notice (or null) and report_draft.
Asks a person when
The incident is SEV1, affects an operating site, or the mitigation is a rollback.
Settings
Your limits, set by your managers.
| Setting | Default | What it controls |
|---|---|---|
| SEV1 at error rate (%) | 5 | Or any outage of a safety-relevant service. |
| Roll back itself | Off | Rollbacks need the incident commander. |
| Notify sites within (minutes) | 10 | When permit to work or emergency notification is affected. |
| Postmortem for | SEV1 and SEV2 | Which incidents get one. |
Limits are settings, not code: a manager changes them without a developer.
Rules it always follows
- Rank causes with evidence from alerts, logs and changes.
- Make any outage of permit to work or emergency notification SEV1, and tell site operations within 10 minutes to fall back to the paper permit procedure.
- Never roll back when off is off.
- Draft a postmortem for SEV1 and SEV2.
See the Incident Response Agent agent on your own work.
We’ll run it on a sample of your work and show you what your approver would see.