Measure what your AI gets right

We test your AI on past cases before launch and check its live work every day after, whether you built it or bought it.

Support AI checks, multi-brand online gaming group

53% claimed to 36%

of support conversations resolved by the AI, checked every morning. The client stopped paying for and staffing around resolutions that hadn't happened, and sized its team for the real queue.

Read the support AI case study

What independent checks found

When a vendor reports its own results, your bill and your staffing plan rest on its number. Checked results let you pay for what the AI resolves and staff for the queue that's left.

Multi-brand online gaming groupSupport AI and release checks, before and after

Support AI, first week of daily checks

Vendor's count53%
Checked by us36%

About 38 conversations the vendor counted as resolved still went to an agent that week.

  • 223conversations checked in the first week
  • 288 of 297checks that matched the verdict a person gave
  • 165real customer cases tested before launch
  • 6brands checked every weekday

Release checks, at handover

Tester check2 days
Automated check18 min

No waiting on a tester pass before a release. Testers spend the time on new features.

  • 39 of 42checklist items automated for brand 1
  • 28 of 37checklist items automated for brand 2
  • 4releases a week across the 2 brands
  • 3device types checked: desktop, Android and iPhone

What changes for your business

The support AI, judged on the vendor's own number before, and on our daily checks after.

MeasureBeforeAfter
What the bill is based onThe vendor's own countResolutions that happened
What the agent rota is sized onThe vendor's own countThe queue left for agents
Conversations checkedSpot checks, with no agreed rulesEvery conversation, every morning
Counted resolutions that hadn't happenedUnknownFound and left out each morning

Testing we've delivered for customers

The support checks and the release checks are both live.

  1. Daily checks on a support AI

    Real customer cases tested before launch, then every live conversation checked each morning. Leadership sees the checked resolution rate next to the vendor's.

    Works with The vendor's help desk, company sign-in
  2. Automated release checks

    Every item on the release checklist that a browser can check runs on its own in the client's build pipeline, with a pass or fail for each item.

    Works with Jenkins, Playwright
  3. Checks on the live sites

    A read-only set of checks against the live sites, so a broken sign-in shows up before customers report it.

    Works with Jenkins, the live sites
    Runs every 15 minutes
  4. Tests from recordings

    Testers record a flow in Chrome and turn it into a test, so the QA team adds to the checks without us.

    Works with Chrome, Playwright
    Handed to the client's QA team in week 2

What automated testing is worth

2 brands release twice a week, and each release waited on about 2 tester-days of checks.

  • Week 1first checks running in the client's pipeline
What automated testing is worth: what is counted, how, and the result
What we countHowResult
Tester check per releaseAbout 2 tester-days2 days
Releases each weekTwice a week, for each of 2 brands4 releases
Tester time on repeat checks4 releases at 2 days8 tester-days a week
Over a working year8 tester-days for 50 weeks, or 3,200 hours400 tester-days
Tester time back each year400 days at €350 a day€140,000

How we deliver it

  1. 01

    Agreed rules first

    What counts as resolved, wrong or unsafe, written down with your team before checking starts.

  2. 02

    Clear cases decided by rules

    A conversation that went to an agent or came back is decided by a rule. AI judges the rest.

  3. 03

    Checked against your team

    Our checks are compared with cases your team already judged before any number counts.

  4. 04

    Personal data masked

    Before any conversation is checked, with every access logged.

Does it work with AI we bought?

Yes. We check results from the outside, so it works for a vendor's bot as well as for systems you or we built.

Why was the checked rate lower than the vendor's?

The vendor counted every conversation the AI kept away from an agent. We counted only the ones where the customer's problem was solved. That's the number the bill is checked against.

Does release automation replace the testers?

No. It takes the repeat checks off them, so they spend the time on new features and the cases a script can't judge.

Pay for the results your AI delivers

Tell us what you run and how it's billed. On a 30-minute call we'll show you how we'd check it and what the first week would tell you.

Ask your AI assistant about us

Open our summary in the assistant you use, or copy the prompt.

Read the prompt