UAE · ENBook a demo
HotelProductDecision-maker

Experimentation Agent

Designs A/B tests and reads the results.

Experimentation AgentDiscovery and Delivery
Industry
Hotel
Department
Product
Team
Discovery and Delivery
Kind
Decision-maker
Approver
Director of Product
Code
PRD-11

It designs A/B tests on the booking engine and guest app, such as upsell placement, room photo order or a breakfast add-on default, and reads the results. It decides whether to roll the winner out and sets the rollout in the experimentation tool once approved.

How it works

7 steps, and a person signs off before step 7.

  1. 01Looks it up

    Finds the record the request is about in the experimentation tool: variant results and guardrail metrics..

  2. 02Reads the request

    Works out what is being asked and what a complete answer needs.

  3. 03Checks the facts

    Checks figures, dates and rules against the source records, not from memory.

  4. 04Forecasts, when it decides

    Projects what each option will cost or return before it recommends one.

  5. 05Works it through

    Prepares the draft, match, recommendation or reply, with its sources.

  6. 06Asks a person

    Hands the work to Director of Product with its reasoning before anything goes out.

  7. 07Acts

    Once approved, it acts in the experimentation tool: it sets the rollout step for the winning variant. and records what was done.

What it works with

Your knowledge, your systems, your people.

Knows

The experimentation standard: significance, power, guardrails (booking conversion, cancellations, average booking value, complaints), and rollout steps.

Reads from

the experimentation tool: variant results and guardrail metrics.

Acts in

the experimentation tool: it sets the rollout step for the winning variant.

Hands back

An object with decision_type (roll_out, extend or stop), lift_percent, p_value, guardrails (a list), next_step_percent and reason.

Asks a person when

A winner would roll out, or a guardrail is breached.

ApproverDirector of Product

Settings

Your limits, set by your managers.

SettingDefaultWhat it controls
Significance level0.05Results count below this.
Power0.8Sample is sized to this.
Longest test (days)28Tests stop after this.
Guardrail tolerance (%)2A guardrail worse by more than this blocks rollout.
Rollout step (%)25Winners roll out in these steps.

Limits are settings, not code: a manager changes them without a developer.

Rules it always follows

  • Size tests for power 0.8 at significance 0.05; stop by 28 days.
  • Block rollout if booking conversion, cancellations or average booking value worsen by more than 2 percent.
  • Roll out in 25 percent steps.
  • Never default a paid add-on to selected where consumer law requires opt-in.

See the Experimentation Agent agent on your own work.

We’ll run it on a sample of your work and show you what your approver would see.

Experimentation Agent · Hotel | Basinwright Industries