Measure what your AI gets right
We test your AI on past cases before launch and check its live work every day after, whether you built it or bought it.
53% claimed to 36%
of support conversations resolved by the AI, checked every morning. The client stopped paying for and staffing around resolutions that hadn't happened, and sized its team for the real queue.
Read the support AI case studyWhat independent checks found
When a vendor reports its own results, your bill and your staffing plan rest on its number. Checked results let you pay for what the AI resolves and staff for the queue that's left.
Support AI, first week of daily checks
About 38 conversations the vendor counted as resolved still went to an agent that week.
- 223conversations checked in the first week
- 288 of 297checks that matched the verdict a person gave
- 165real customer cases tested before launch
- 6brands checked every weekday
Release checks, at handover
No waiting on a tester pass before a release. Testers spend the time on new features.
- 39 of 42checklist items automated for brand 1
- 28 of 37checklist items automated for brand 2
- 4releases a week across the 2 brands
- 3device types checked: desktop, Android and iPhone
What changes for your business
The support AI, judged on the vendor's own number before, and on our daily checks after.
| Measure | Before | After |
|---|---|---|
| What the bill is based on | The vendor's own count | Resolutions that happened |
| What the agent rota is sized on | The vendor's own count | The queue left for agents |
| Conversations checked | Spot checks, with no agreed rules | Every conversation, every morning |
| Counted resolutions that hadn't happened | Unknown | Found and left out each morning |
Testing we've delivered for customers
The support checks and the release checks are both live.
Daily checks on a support AI
Real customer cases tested before launch, then every live conversation checked each morning. Leadership sees the checked resolution rate next to the vendor's.
Works with The vendor's help desk, company sign-inAutomated release checks
Every item on the release checklist that a browser can check runs on its own in the client's build pipeline, with a pass or fail for each item.
Works with Jenkins, PlaywrightChecks on the live sites
A read-only set of checks against the live sites, so a broken sign-in shows up before customers report it.
Works with Jenkins, the live sitesRuns every 15 minutesTests from recordings
Testers record a flow in Chrome and turn it into a test, so the QA team adds to the checks without us.
Works with Chrome, PlaywrightHanded to the client's QA team in week 2
What automated testing is worth
2 brands release twice a week, and each release waited on about 2 tester-days of checks.
- Week 1first checks running in the client's pipeline
| What we count | How | Result |
|---|---|---|
| Tester check per release | About 2 tester-days | 2 days |
| Releases each week | Twice a week, for each of 2 brands | 4 releases |
| Tester time on repeat checks | 4 releases at 2 days | 8 tester-days a week |
| Over a working year | 8 tester-days for 50 weeks, or 3,200 hours | 400 tester-days |
| Tester time back each year | 400 days at €350 a day | €140,000 |
How we deliver it
- 01
Agreed rules first
What counts as resolved, wrong or unsafe, written down with your team before checking starts.
- 02
Clear cases decided by rules
A conversation that went to an agent or came back is decided by a rule. AI judges the rest.
- 03
Checked against your team
Our checks are compared with cases your team already judged before any number counts.
- 04
Personal data masked
Before any conversation is checked, with every access logged.
Questions about AI testing
Related reading
Does it work with AI we bought?
Yes. We check results from the outside, so it works for a vendor's bot as well as for systems you or we built.
Why was the checked rate lower than the vendor's?
The vendor counted every conversation the AI kept away from an agent. We counted only the ones where the customer's problem was solved. That's the number the bill is checked against.
Does release automation replace the testers?
No. It takes the repeat checks off them, so they spend the time on new features and the cases a script can't judge.
Pay for the results your AI delivers
Tell us what you run and how it's billed. On a 30-minute call we'll show you how we'd check it and what the first week would tell you.
Other services
- AI workflow automationTime saved on data entry, checks and follow-ups
- AI integration servicesAI in the tools your team already uses
- Legacy system modernisationReplace rented and outgrown systems
- Private AI and AI cost controlOne gateway, team budgets, data kept private
- AI consulting and enablementGoals, tested tools and training for each team
- AI governanceOwners, permissions and records for AI in use