Playwright merge-reports: combine sharded blob reports into HTML, JUnit or JSON

Why sharded runs need merging

With --shard=1/4 … --shard=4/4 every machine runs a part of the suite and writes its own report. Four HTML reports mean four downloads and no single place that shows the whole run. Playwright solves this with the blob reporter: a zip with every event of the run, including attachments such as traces and screenshots, that can be turned into any other report later.

Step 1: write a blob report on each shard

Use the blob reporter in CI and keep the HTML reporter for local runs, as Playwright's sharding guide does:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  reporter: process.env.CI ? 'blob' : 'html',
});

By default the report goes to blob-report/ next to package.json. The file is named report-<hash>-<shard_number>.zip when sharding, so files from different shards never clash. The options outputDir, fileName and outputFile (or PLAYWRIGHT_BLOB_OUTPUT_DIR, PLAYWRIGHT_BLOB_OUTPUT_NAME, PLAYWRIGHT_BLOB_OUTPUT_FILE) change that. A custom fileName must end in .zip; Playwright 1.63 refuses anything else with "Blob report file name must end with .zip extension".

Step 2: collect the blobs in one folder

Upload blob-report/ from every shard as a CI artifact, and upload it even when tests fail. On GitHub Actions that is if: ${{ !cancelled() }} on the upload step. Then, in a job that runs after all shards, download every blob into a single folder. With actions/download-artifact, pattern: blob-report-* plus merge-multiple: true puts all zips side by side:

  merge-reports:
    if: ${{ !cancelled() }}
    needs: [playwright-tests]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v6
        with:
          node-version: lts/*
      - run: npm ci
      - uses: actions/download-artifact@v5
        with:
          path: all-blob-reports
          pattern: blob-report-*
          merge-multiple: true
      - run: npx playwright merge-reports --reporter html ./all-blob-reports
      - uses: actions/upload-artifact@v4
        with:
          name: html-report--attempt-${{ github.run_attempt }}
          path: playwright-report
          retention-days: 14

This is the workflow from Playwright's sharding guide. The full version with the sharded test job is in the GitHub Actions setup guide; for GitLab CI, Azure Pipelines, CircleCI and Jenkins see sharding on other CI systems.

Step 3: merge into the reports you need

The basic merge produces a normal HTML report in playwright-report/, which npx playwright show-report opens:

npx playwright merge-reports --reporter html ./all-blob-reports

--reporter accepts several reporters separated by commas. This gives you an HTML report for people and GitHub annotations in the same step:

npx playwright merge-reports --reporter=html,github ./all-blob-reports

Merging into JUnit or JSON

JUnit and JSON print to standard output unless you give them a file. Either set the environment variable for the file name, or pass a merge config with -c. The environment variables are the same ones the reporters use in a normal run:

PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml npx playwright merge-reports --reporter=junit ./all-blob-reports
PLAYWRIGHT_JSON_OUTPUT_NAME=results.json npx playwright merge-reports --reporter=json ./all-blob-reports

A merge config is clearer when you want several reports with options. It can differ from the config the shards ran with:

// merge.config.ts
export default {
  testDir: 'e2e',
  reporter: [
    ['html', { open: 'never' }],
    ['junit', { outputFile: 'results.xml' }],
    ['json', { outputFile: 'results.json' }],
  ],
};

npx playwright merge-reports --config=merge.config.ts ./all-blob-reports

The JUnit reporter guide covers the JUnit options, including how retries and flaky tests appear in the XML.

Errors you may see

No report files found in ./all-blob-reports

In Playwright 1.63, merge-reports lists the folder you pass and keeps only the files ending in .zip. It does not look into subfolders. If each artifact was downloaded into its own subfolder (all-blob-reports/blob-report-1/report-….zip), the folder has no zips at its top level and the merge stops. Download with merge-multiple: true, or move the zips up one level before merging. Also check that the shard jobs actually uploaded something: a shard that failed before the upload step leaves nothing to download.

Blob reports being merged were recorded with different test directories

Every blob records the root folder of the tests. If they differ, for example because shards ran on Linux and Windows or with different configs, Playwright cannot tell which paths to use and stops. Its error message suggests the fix: pass a merge config with -c whose testDir points to the real test location. The sharding guide gives the same advice for reports from different operating systems.

Blob report … was created with a newer version of Playwright

The job that merges uses an older Playwright than the shards that wrote the blobs. Run npm ci in the merge job so npx playwright resolves to the version in your lockfile, the same one the shards used.

Running the same tests in several environments

If you run the same tests against several environments rather than splitting them, give each environment a tag with the tag config option, for example tag: process.env.CI_ENVIRONMENT_NAME. Playwright's docs say the blob report and the merge tool pick it up, so the merged report shows which environment each result came from, and the tag is part of the blob file name, so the files do not overwrite each other.

What a merged report cannot show

Merging works with what was uploaded. If a shard runner was killed, for example out of memory with exit code 137, it may never write its blob. The merged report then shows the other shards, all green, and the killed shard's tests are not listed at all. Nothing in the report says that anything is missing. Compare the test count with a previous run, or see how to find tests missing after a crashed shard.

A merged report also describes one run. Whether a failure is new on this pull request, already failing on main, or a test that has been flaky for weeks are questions about many runs.

TurnSignal takes a different route for sharded runs: each shard sends its results as it runs, and the run page combines them, so there is no merge job to maintain. Tests a crashed shard never reported are listed as Missing instead of disappearing, and each failure is labelled new, also failing on main or known flaky. It is one more reporter next to the ones you have; see the docs.