Shipping the Product Wasn't Enough. It Needed Evidence.

A successful deployment proves that the deployment worked. It does not prove that the product still works.

This should not be a controversial statement. Yet plenty of teams treat a green deployment as if it has earned a small quality certificate and a round of applause.

It has not.

Building Soccer, a platform intended to support people who coach children’s football as volunteers, made one thing clear: getting the application live was an important milestone, not the end of the work. Product quality could not depend on somebody remembering which journeys to click through after every change.

Manual checking has its place. It is also remarkably easy to postpone when everyone is busy and the change looks small. And, as software has repeatedly taught us, “small change” is often another name for “interesting incident later.”

The next problem was therefore an engineering problem: how can a team build confidence that the application still behaves as expected when it changes?

The Product Was Only Half the Work

Soccer is deployed with Vercel and uses Supabase, with GitHub supporting the development workflow. It is a real application built around a real use case, not a test site created purely to make a dashboard look busy.

That distinction matters. Test automation has more value when it protects a product that people can actually use. The purpose is not to collect green checks. The purpose is to detect when a change breaks an important user journey before we learn about it the expensive way.

Once the core application was functional, it needed a repeatable way to be validated from the outside, as a user would experience it. That meant end-to-end testing.

Keeping the Responsibilities Separate

The application, the Playwright test suite, and the published test report do not all live in one repository.

The application has its own repository. The end-to-end tests live in a separate repository. The public reporting page lives in another repository and is served through GitHub Pages.

That separation is deliberate.

The application repository is where product behaviour changes. The test repository is where checks protect those behaviours. The reporting site is where anyone can review the resulting evidence without needing to clone a project, run a command locally, or ask someone to translate a terminal log.

Separating those responsibilities does not make the system magically perfect. Nothing does. But it makes the boundaries explicit, and explicit boundaries are much easier to maintain than a collection of assumptions living in one increasingly crowded repository.

The Feedback Loop

When a commit reaches the application branch, an automated workflow triggers the Playwright end-to-end tests.

The tests validate the application through the user-facing interface. After the run, the workflow generates an Allure report and publishes it to GitHub Pages:

View the public E2E test report

The flow is straightforward:

Commit to the application branch
        ↓
Playwright end-to-end tests run
        ↓
Allure report is generated
        ↓
Report is published to GitHub Pages
        ↓
The latest quality evidence is available to review

There is no grand theory hidden inside this architecture. That is part of the point. Useful engineering is often less about assembling the most fashionable collection of tools and more about designing a clear path from a change to evidence.

In this case, the tools are Vercel, Supabase, GitHub, Playwright, Allure, and GitHub Pages. But the important outcome is the feedback loop they create.

What the Suite Covers Today

The current suite contains six automated checks, executed across three browsers. It is intentionally focused on the journeys and product surface that provide the most useful early signal when the application changes.

The checks cover the landing-page content, account creation, and login. Those journeys run through the interface as a user would experience them. The suite also uses the API to confirm that an account has been created and to delete the test account afterwards. That gives the run a reliable way to verify the state it created and to leave the environment clean for the next execution.

This is not a claim that every behaviour is covered. It is a visible, repeatable starting point around the paths where a regression would quickly affect a new or returning user.

Why a Public Report Matters

An automated suite that runs somewhere but cannot explain its result is only partially useful.

“The pipeline failed” is not an insight. It is an invitation to begin an investigation.

Allure gives the test execution a usable reporting layer. Publishing it through GitHub Pages makes the latest results accessible without treating quality information as a private artifact reserved for whoever happens to have repository access and enough patience to find the correct workflow run.

That visibility matters for more than convenience:

• It creates a shared reference point for the current automated checks.
• It makes failures easier to review as evidence rather than as vague status messages.
• It reinforces that quality is part of the product’s engineering system, not an activity that appears just before release.

A public report does not replace communication, exploratory testing, code review, or engineering judgement. A dashboard is not a strategy. A green dashboard is definitely not a strategy.

But visible evidence makes better conversations possible. It gives people something concrete to inspect when deciding whether a change is safe enough to move forward.

This Is What Hands-On QA Leadership Looks Like to Me

QA leadership is sometimes described as if it begins when you stop building things and start attending meetings about building things.

That is not a useful definition.

Leadership does involve setting direction, making trade-offs, communicating risk, and helping people work well together. But technical credibility also comes from understanding the system closely enough to improve it.

For this project, that meant taking ownership of more than the test cases:

• Building and deploying a functional application for a defined user need.
• Creating a separate Playwright end-to-end test suite to validate it.
• Connecting code changes to automated feedback.
• Generating readable execution evidence with Allure.
• Publishing that evidence in a place where it can be reviewed easily.

None of these pieces is impressive in isolation. The value is in connecting them into a system that makes the product easier to change with confidence.

Engineering ownership is not about personally writing every line forever. It is about ensuring that important problems are actually solved, including the problems that sit between product delivery, quality feedback, and operational visibility.

Three Lessons Worth Reusing

1. Build the feedback loop, not just the tests

Writing an automated test is only one part of the work. A useful system also needs a trigger, reliable execution, evidence, and a clear route for people to see the outcome.

Without that loop, the suite becomes easy to ignore. And ignored automation is just code with excellent intentions.

2. Treat reports as a product for your engineering team

The people reading a test result need answers: what ran, what failed, and what evidence is available? A report that only says “failed” transfers the real work to somebody else.

Make the output easy to find and practical to use. If quality information takes twenty minutes to locate, it will somehow always be located after the decision has already been made.

3. A leadership role should not disconnect you from engineering reality

You do not need to be the person who implements every detail. But you should be able to reason about the architecture, understand the failure modes, and contribute to the systems that improve delivery.

That is the kind of technical leadership that matters: close enough to the work to make better decisions, and intentional enough to turn good engineering practices into repeatable capability.

The Work After “It Works”

The platform is live. The end-to-end tests run when the application changes. The latest Allure report is publicly available.

That does not mean the work is complete. Quality is not a box that gets checked because a report has been published. It is a capability we continue to build as the product, its risks, and its users evolve.

But building the product and building the confidence to change it are two different jobs. Doing only the first one is how teams end up surprised by their own releases.

How does your team turn code changes into quality evidence that people can actually use?