Quality assurance, online gaming

Releases no longer wait on testers

We automated the release checklist in the client's own build pipeline, so releases go out when the code is ready.

Client
A multi-brand online gaming group
Teams
Engineering and QA
Runs in
The client's Jenkins, on Playwright
Book a 30-minute call
18 minfor the release check, down from 2 tester-days
400 daysof tester time a year back from repeat checks
1 weekto the first checks running in the pipeline
2 weeksto handover to the client's QA team

What changed for the business

What it means for the group

  • The tester time freed each year is worth about €140,000 at €350 a day. It now goes to new features and the cases a script can't judge.
  • The checks run in the client's own pipeline and repository, and their QA team extends them, so they keep running without us.
MeasureBeforeAfter
Time for the release checkAbout 2 tester-days18 minutes
Checklist items run automatically on stagingChecked by testers39 of 42 and 28 of 37
Checks on the live sitesNoneEvery 15 minutes, read-only

We measured coverage and run time at handover. The tester-days a year come from 4 releases a week at 2 tester-days each, over 50 weeks.

What the group needed

The group wanted to release faster without cutting the checks. Each release had a checklist of 37 to 42 items per brand, from sign-in and account top-up to responsible gaming text and cookie consent, and testers worked through it item by item.

How it worked before

  • Every release waited on about 2 tester-days of checks. At 2 releases a week per brand, that was about 8 tester-days a week across the 2 brands.
  • The team had been working on automating the checklist for about 2.5 years, and releases still waited on the tester pass.

What we built

  1. 01

    The checklist, automated

    Every checklist item a browser can check runs as an automated test in the client's Jenkins, with a pass or fail per item.

  2. 02

    Desktop, Android and iPhone

    The same checks on the devices customers use, including responsible gaming text checked against the content system.

  3. 03

    Checks on the live sites

    A read-only set of checks against the live sites every 15 minutes, so a broken sign-in shows up before customers report it.

  4. 04

    Tests from recordings

    Testers record a flow in Chrome and turn it into a test, so the QA team adds to the checks themselves.

How it went live

  1. Week 1

    The first release checks running in the build pipeline.

  2. Week 2

    Coverage on staging reached 39 of 42 items for one brand and 28 of 37 for the other, with documentation for the QA team.

  3. End of week 2

    Handed over to the client's QA team, who run and extend it.

Impressive progress, given the timeline. The release checklist used to take our testers days. Within the first week it was running in our own pipeline, and by the end of the second our QA team was running and extending it themselves.

Director of EngineeringOnline gaming group

Questions about this project

Does this replace the testers?

No. It takes the repeat checks off them, so they spend their time on new features and the cases a script can't judge.

Where does it run?

In the client's own build server and repository. Nothing runs on our side.

Can the QA team change it?

Yes. They record new flows in Chrome, and the documentation covers every job and setting.

Find the tester-days your release checklist costs

Bring your release checklist to a 30-minute call. We'll show you which items can run in your pipeline, and how many tester-days that frees.

Ask your AI assistant about us

Open our summary in the assistant you use, or copy the prompt.

Read the prompt