- Industry
- IT Company
- Department
- Engineering
- Team
- Platform and Reliability
- Kind
- Investigator
- Approver
- Delivery Director
- Code
- ENG-19
It designs and runs load, stress and soak tests for the SaaS products and for client systems before go-live, compares results with the non-functional requirements in the SOW and the last baseline, and explains each bottleneck. It reports whether the system meets the contracted performance criteria, never load-testing a production system without the client's written approval. **The problem it solves:** Why the system does or does not meet its performance targets under load: which component limits it and what the evidence shows.
How it works
6 steps, and a person signs off before step 6.
- 01Looks it up
Finds the record the request is about in the cloud monitoring and observability stack: load test results, resource metrics during the test and the previous baseline..
- 02Reads the request
Works out what is being asked and what a complete answer needs.
- 03Checks the facts
Checks figures, dates and rules against the source records, not from memory.
- 04Works it through
Weighs competing explanations against the evidence and shows which one holds up.
- 05Asks a person
Hands the work to Delivery Director with its reasoning before anything goes out.
- 06Acts
Once approved, it acts in the project management tool: it files the performance report against the release and opens fixes for bottlenecks. and records what was done.
What it works with
Your knowledge, your systems, your people.
Knows
The performance testing guide: non-functional requirements per engagement, load profiles, baselines, test environments and the rules for testing client systems.
Reads from
the cloud monitoring and observability stack: load test results, resource metrics during the test and the previous baseline.
Acts in
the project management tool: it files the performance report against the release and opens fixes for bottlenecks.
Hands back
An object with verdict (meets, fails or regression), results (a list with scenario, metric, value and target), bottlenecks (a ranked list with component, evidence and fix) and baseline_delta_percent.
Asks a person when
The system fails a contracted performance criterion before client acceptance, or a test would run against production.
Settings
Your limits, set by your managers.
| Setting | Default | What it controls |
|---|---|---|
| Default p95 budget (ms) | 400 | Used where the SOW names no target. |
| Error rate budget (%) | 1 | Errors above this under load fail. |
| Regression tolerance (%) | 10 | A slowdown against baseline above this is a regression. |
| Headroom over peak | 2 | The system must hold this multiple of expected peak. |
| May load production | Off | When off, tests never run against production. |
Limits are settings, not code: a manager changes them without a developer.
Rules it always follows
- Judge results against the SOW's non-functional requirements, or 400 ms and 1 percent errors when none are given.
- Call a slowdown over 10 percent against baseline a regression.
- Test to 2 times expected peak.
- Never load production unless off is true and the client approved in writing.
- Explain each bottleneck with the evidence that locates it.
See the Performance Testing Agent agent on your own work.
We’ll run it on a sample of your work and show you what your approver would see.