- Industry
- Procure AI
- Department
- Engineering
- Team
- Platform and Reliability
- Kind
- Investigator
- Approver
- Release Manager
- Code
- ENG-19
It designs and runs load, stress and soak tests against a test environment for the platform's peak moments, such as month-end invoice surges, a large reverse auction with hundreds of suppliers bidding at once and mass approval notifications, and pins each bottleneck on the tier and resource that saturates first. It reports whether the release meets its budgets and what to fix first, and never runs load against production or a client's ERP. **The problem it solves:** Why the platform misses its performance budget under load: the scenario, the load at which latency or errors climb, the resource that saturates first, and what changed since the last baseline.
How it works
6 steps, and a person signs off before step 6.
- 01Looks it up
Finds the record the request is about in the load-testing tool: the test runs, latency and error series, and resource metrics recorded for the service and baseline named..
- 02Reads the request
Works out what is being asked and what a complete answer needs.
- 03Checks the facts
Checks figures, dates and rules against the source records, not from memory.
- 04Works it through
Weighs competing explanations against the evidence and shows which one holds up.
- 05Asks a person
Hands the work to Release Manager with its reasoning before anything goes out.
- 06Acts
Once approved, it acts in the load-testing tool: it schedules the test run against the environment named and attaches the report to the release. and records what was done.
What it works with
Your knowledge, your systems, your people.
Knows
The performance standards: latency and error budgets per journey, expected traffic and the month-end and auction peaks, test environments and their limits, the synthetic tenant data set and past baselines.
Reads from
the load-testing tool: the test runs, latency and error series, and resource metrics recorded for the service and baseline named.
Acts in
the load-testing tool: it schedules the test run against the environment named and attaches the report to the release.
Hands back
An object with verdict (pass, regression or fail), scenarios (a list, each with name, max_rps_held, p95_ms, p99_ms, error_rate_percent and budget_met true or false), bottlenecks (a ranked list, each with tier, resource, evidence, confidence and fix), baseline_delta_percent (or null) and next_tests (a list).
Asks a person when
The test would run against production or a shared environment, the release misses its budget and the request asks to ship anyway, or an external endpoint cannot be stubbed.
Settings
Your limits, set by your managers.
| Setting | Default | What it controls |
|---|---|---|
| Latency budget (p95, ms) | 400 | Used for a journey whose own budget is not in the standards. |
| Bid acknowledgement budget (p99, ms) | 1000 | In a live auction every bid must be accepted and ranked within this, or suppliers are treated unfairly. |
| Error rate budget under load (%) | 1 | A scenario above this error rate at target load fails. |
| Regression tolerance (%) | 10 | A p95 more than this above the baseline is a regression. |
| Headroom over expected peak | 2 | The system must hold this multiple of the month-end peak within budget. |
| May load-test production | Off | When off, load runs only against test environments. |
Limits are settings, not code: a manager changes them without a developer.
Rules it always follows
- A scenario passes when it holds 2 times the month-end peak with p95 within budget, falling back to 400 ms, and errors under 1 percent.
- An auction scenario passes only when every bid is acknowledged within 1000 ms at p99 and ranked in the order received.
- A p95 more than 10 percent above the baseline is a regression, even within budget.
- Pin each bottleneck on the first resource to saturate, with the metric and load at which it did, and say what would confirm it.
- Run load only against test environments unless off is true and the Release Manager agrees, and stub every client ERP, bank and e-invoicing endpoint.
- Rank fixes by load gained per unit of effort.
See the Performance Testing Agent agent on your own work.
We’ll run it on a sample of your work and show you what your approver would see.