Quality assurance, online gaming
Releases no longer wait on testers
We automated the release checklist in the client's own build pipeline, so releases go out when the code is ready.
- Client
- A multi-brand online gaming group
- Teams
- Engineering and QA
- Runs in
- The client's Jenkins, on Playwright
- Service
- AI testing and measurement →
What changed for the business
What it means for the group
- The tester time freed each year is worth about €140,000 at €350 a day. It now goes to new features and the cases a script can't judge.
- The checks run in the client's own pipeline and repository, and their QA team extends them, so they keep running without us.
| Measure | Before | After |
|---|---|---|
| Time for the release check | About 2 tester-days | 18 minutes |
| Checklist items run automatically on staging | Checked by testers | 39 of 42 and 28 of 37 |
| Checks on the live sites | None | Every 15 minutes, read-only |
We measured coverage and run time at handover. The tester-days a year come from 4 releases a week at 2 tester-days each, over 50 weeks.
What the group needed
The group wanted to release faster without cutting the checks. Each release had a checklist of 37 to 42 items per brand, from sign-in and account top-up to responsible gaming text and cookie consent, and testers worked through it item by item.
How it worked before
- Every release waited on about 2 tester-days of checks. At 2 releases a week per brand, that was about 8 tester-days a week across the 2 brands.
- The team had been working on automating the checklist for about 2.5 years, and releases still waited on the tester pass.
What we built
- 01
The checklist, automated
Every checklist item a browser can check runs as an automated test in the client's Jenkins, with a pass or fail per item.
- 02
Desktop, Android and iPhone
The same checks on the devices customers use, including responsible gaming text checked against the content system.
- 03
Checks on the live sites
A read-only set of checks against the live sites every 15 minutes, so a broken sign-in shows up before customers report it.
- 04
Tests from recordings
Testers record a flow in Chrome and turn it into a test, so the QA team adds to the checks themselves.
How it went live
- Week 1
The first release checks running in the build pipeline.
- Week 2
Coverage on staging reached 39 of 42 items for one brand and 28 of 37 for the other, with documentation for the QA team.
- End of week 2
Handed over to the client's QA team, who run and extend it.
Impressive progress, given the timeline. The release checklist used to take our testers days. Within the first week it was running in our own pipeline, and by the end of the second our QA team was running and extending it themselves.
Questions about this project
Does this replace the testers?
No. It takes the repeat checks off them, so they spend their time on new features and the cases a script can't judge.
Where does it run?
In the client's own build server and repository. Nothing runs on our side.
Can the QA team change it?
Yes. They record new flows in Chrome, and the documentation covers every job and setting.
More case studies
- Client support, online gamingStaff stopped copying client emails into Jira0 client emails staff copy into the tracker, down from every one
- Customer support, online gamingSpend and rota set on real resolutions36% of conversations resolved by the AI, against 53% claimed by the vendor
- Human resources5 hours a week back for HR, payroll on time250 hrs a year of HR time back from reports and chasing
- IT servicesEnquiries answered in 15 minutes15 min median time to a reviewed reply, down from 1 to 2 days
Find the tester-days your release checklist costs
Bring your release checklist to a 30-minute call. We'll show you which items can run in your pipeline, and how many tester-days that frees.