UAE · ENBook a demo
Procure AIProductDecision-maker

Experimentation Agent

Designs A/B tests and reads the results.

Experimentation AgentDiscovery and Delivery
Industry
Procure AI
Department
Product
Team
Discovery and Delivery
Kind
Decision-maker
Approver
Head of Product
Code
PRD-11

It designs A/B tests on the platform, such as a new guided-buying search that steers requisitioners to catalogues, a reworded supplier onboarding form or an approval reminder cadence, and reads the results: ship, iterate, stop or keep running. It forecasts the effect of rolling a winner out on maverick spend, PO cycle time or onboarding completion and, once approved, sets the next rollout step on the feature flag.

How it works

7 steps, and a person signs off before step 7.

  1. 01Looks it up

    Finds the record the request is about in the spend analytics warehouse: experiment exposure and metric results by variant and client..

  2. 02Reads the request

    Works out what is being asked and what a complete answer needs.

  3. 03Checks the facts

    Checks figures, dates and rules against the source records, not from memory.

  4. 04Forecasts, when it decides

    Projects what each option will cost or return before it recommends one.

  5. 05Works it through

    Prepares the draft, match, recommendation or reply, with its sources.

  6. 06Asks a person

    Hands the work to Head of Product with its reasoning before anything goes out.

  7. 07Acts

    Once approved, it acts in the source-to-pay suite: it sets the next rollout percentage on the feature flag. and records what was done.

What it works with

Your knowledge, your systems, your people.

Knows

The experimentation standards, metric definitions (catalogue share, maverick-spend share, PO cycle time, onboarding completion), guardrails, client consent to experiments and past test results.

Reads from

the spend analytics warehouse: experiment exposure and metric results by variant and client.

Acts in

the source-to-pay suite: it sets the next rollout percentage on the feature flag.

Hands back

An object with hypothesis, design (an object with metric, sample_size and duration_days, or null), verdict (ship, iterate, stop or keep_running, or null), effect_percent, guardrails (a list with metric and change_percent) and next_rollout_percent.

Asks a person when

The verdict ships or rolls out a variant, a guardrail is worse by more than 1 percent, or the test touches approvals, payments or supplier data.

ApproverHead of Product

Settings

Your limits, set by your managers.

SettingDefaultWhat it controls
Significance level0.05The threshold for a significant result.
Smallest effect to detect (%)3Sample size is set to detect this.
Longest test (days)35Long enough to cover a month-end cycle.
Guardrail metricsapproval cycle time, invoice exception rate, supplier onboarding drop-offMetrics that must not get worse.
Guardrail tolerance (%)1A guardrail worse by more than this stops a ship.
Rollout step (%)25The share of users added at each rollout step.

Limits are settings, not code: a manager changes them without a developer.

Rules it always follows

  • Tests run only on clients whose contract allows experiments; approval steps, limits and controls are never varied.
  • Size the test to detect 3 percent at 0.05 and run at most 35 days, spanning a month-end.
  • A variant ships only when significant and no guardrail in approval cycle time, invoice exception rate, supplier onboarding drop-off is worse by more than 1 percent.
  • Roll out in steps of 25 percent of users.
  • State the forecast effect on the client KPI, with its range.

See the Experimentation Agent agent on your own work.

We’ll run it on a sample of your work and show you what your approver would see.

Experimentation Agent · Procure AI | Basinwright Industries