Playwright test reporting in CI: the HTML report, and when you need more

Playwright's built-in reporters

You choose reporters with the reporter option in playwright.config.ts or --reporter on the command line. The built-in ones:

• list, line and dot: progress in the terminal, from most to least detailed. The dot reporter marks a test that passed on retry (flaky) with its own symbol.
• html: a self-contained report you open in a browser, with steps, errors, screenshots, videos and traces.
• blob: everything about a run, including attachments, in a format made for merging (see sharding below).
• json and junit: files for other tools. Many CI systems can display JUnit XML in their own test tab.
• github: failure annotations on the lines of code in GitHub Actions.

You can use several at once. A common CI setup prints a compact summary in the log and writes the HTML report for later:

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: process.env.CI
    ? [['dot'], ['html', { open: 'never' }]]
    : [['list'], ['html']],
});

Keeping the HTML report from CI

The HTML report is written to playwright-report/ when the run ends. In CI, upload that folder as a build artifact (for example with actions/upload-artifact on GitHub Actions), download it when you need it, and open it locally with npx playwright show-report. Record traces on the first retry or on failure so the report has something to show when a test fails.

Make failures debuggable from the report

A report is only as useful as the evidence attached to a failed test. Playwright decides what to record through the use options, and these settings keep CI fast while giving you something to look at when a test fails:

use: {
  trace: 'on-first-retry',        // or 'retain-on-failure'
  video: 'retain-on-failure',
  screenshot: 'only-on-failure',
},

The trace is the most valuable of the three: it lets you step through every action with the DOM, network requests and console at that moment. Videos help when the failure is visual or timing-related, and screenshots are the quickest look. Remember that all three show your application as it was during the test, including any test data on screen, so treat the report artifact as you would any other build output that may contain data.

Sharding: one report from many machines

With --shard=1/4 … --shard=4/4, each machine runs a quarter of the suite and writes its own report. To get one report, make each shard write a blob report, collect them in one folder, and merge:

# playwright.config.ts: reporter: process.env.CI ? 'blob' : 'html'
npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

# in a final job, after downloading every shard's blob-report into one folder:
npx playwright merge-reports --reporter html ./all-blob-reports

Playwright's sharding guide has the full GitHub Actions workflow: a matrix job per shard that uploads blob-report, and a merge-reports job with needs: and if: ${{ !cancelled() }} so it runs even when a shard fails. For balanced shards, fullyParallel: true splits by test rather than by file.

What the HTML report does well

• Everything about one run in one place: every test, every attempt, errors with code, and the trace to step through.
• No service to run and no data leaves your CI.
• Filters by status, including the tests that were flaky in that run, and a search box.

For a small suite, a few runs a day and one person looking at failures, this is often all you need.

The questions a single report can't answer

As the suite, the team and the number of pull requests grow, the questions after a red run change:

• Did my change break this, or was it already failing on main? A report of your branch has no memory of main.
• Is this test flaky, or is this the first time? A report knows about the retries in this run, not the last hundred runs.
• What happened to the tests on the runner that crashed? If a runner is killed (for example out of memory), its part of the report may never be written, and those tests simply do not appear.
• Who needs to know? Someone has to download and open an artifact before anyone sees a failure.
• Where did this test start failing? Answering that means opening old artifacts one by one, if they have not expired.

Adding history without giving up the HTML report

Reporters stack, so you can keep the HTML report and add one that keeps results across runs. With TurnSignal, that is one more entry in the list:

reporter: [['list'], ['html', { open: 'never' }], ['turnsignal']],

• Every run opens with a plain verdict, and every failure is compared with your default branch: new failure, also failing on main or known flaky.
• Results are written to disk in CI before they are sent. If a runner crashes, the tests it never finished are listed as Missing with the likely reason, instead of silently disappearing.
• Shards of one pipeline appear as one run on GitHub Actions, GitLab CI, Jenkins, CircleCI, Azure Pipelines and Buildkite (on any other CI you set one variable).
• On GitHub, the verdict is posted as a sticky pull request comment. Slack, Discord and webhook alerts, the trace viewer in the browser, and a history for every test are included.
• The reporter never changes your exit code. If our API is unreachable your tests run as usual, and npx turnsignal upload sends the saved results afterwards.

The docs have the CI setup for GitHub Actions, GitLab and other CI.