Playwright JUnit reporter: XML output file, options, retries and flaky tests

Write the report to a file

In playwright.config.ts, give the reporter an output file. Paths are relative to the config file:

// playwright.config.ts
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['list'],
    ['junit', { outputFile: 'test-results/junit.xml' }],
  ],
});

From the command line, the reporter takes its file name from the environment:

PLAYWRIGHT_JUNIT_OUTPUT_NAME=results.xml npx playwright test --reporter=junit

PLAYWRIGHT_JUNIT_OUTPUT_DIR sets the folder for that name, and PLAYWRIGHT_JUNIT_OUTPUT_FILE sets a full path and overrides both. If none of these is set, the XML goes to standard output, mixed with anything else printed there. That is the most common reason for "the JUnit file was never created".

Every option in Playwright 1.63

• outputFile: where to write the XML. Environment: PLAYWRIGHT_JUNIT_OUTPUT_NAME, PLAYWRIGHT_JUNIT_OUTPUT_DIR, PLAYWRIGHT_JUNIT_OUTPUT_FILE.
• stripANSIControlSequences: remove terminal color codes from error text. Turn it on if your CI shows sequences like [31m in test results. Environment: PLAYWRIGHT_JUNIT_STRIP_ANSI.
• includeProjectInTestName: prefix every test name with its project, for example [chromium] login › shows an error. Useful when the same tests run in several browsers and your CI groups results by name. Environment: PLAYWRIGHT_JUNIT_INCLUDE_PROJECT_IN_TEST_NAME.
• includeRetries: record each attempt of a retried test (see below). Environment: PLAYWRIGHT_JUNIT_INCLUDE_RETRIES. It is in the 1.63 types and reporter source, but not yet in the reporter table on playwright.dev.
• omitTags: leave test tags out of the failure details, where they are appended by default. Environment: PLAYWRIGHT_JUNIT_OMIT_TAGS.

Two environment variables have no config option: PLAYWRIGHT_JUNIT_SUITE_ID and PLAYWRIGHT_JUNIT_SUITE_NAME set the id and name attributes of the root <testsuites> element. Both are empty by default.

What the XML contains

The structure, as written by the 1.63 reporter:

• <testsuites>: totals for the whole run, with tests, failures, skipped, errors and time in seconds.
• <testsuite>: one per test file per project. name is the file, hostname is the project name, plus timestamp and the same counters.
• <testcase>: one per test. name joins the describe blocks and the test title with ›; classname is the file.
• <properties>: one <property> per annotation, with the annotation type as name and its description as value. The source notes this follows the format Xray reads, and other tools can use it too.
• <failure> or <error>: why the test failed. A failed expect assertion becomes <failure> with a type such as expect.toBeVisible; any other thrown error becomes <error>, with the error name from the start of its message (such as Error) as type.
• <skipped>: for skipped tests.
• <system-out> and <system-err>: console output of the test. Attachments with a file, such as screenshots and traces, are listed in <system-out> as [[ATTACHMENT|path]], relative to the XML file.

Because assertion failures and other errors are separate elements, CI dashboards that count "failures" and "errors" separately will show a timed-out click or a test timeout as an error, not a failure. That is by design, not a bug in your tests.

Retries and flaky tests

This is the part most teams miss. With retries set and includeRetries off (the default), a test that fails on its first attempt and passes on the second is written as a passing <testcase>. Its time is the sum of all attempts and its output includes every attempt, but there is no failure element. Any tool reading the JUnit file sees a pass. Playwright itself calls that test flaky; your JUnit-based dashboard will never know.

Turn on includeRetries to keep that information:

reporter: [['junit', { outputFile: 'test-results/junit.xml', includeRetries: true }]],

• Flaky test (a later attempt passed): still a pass, with no <failure>, but each failed attempt is added as <flakyFailure> or <flakyError> with its message, stack trace and output. time is the passing attempt.
• Failed test (every attempt failed): <failure> or <error> describes the first attempt, and each later failed attempt is added as <rerunFailure> or <rerunError>. time is the first attempt.

These are the element names Maven Surefire uses for re-run tests, so tools that understand Surefire's flaky tests can read them. Check that your CI's JUnit parser does before you rely on it: a parser that does not know the elements will ignore them, and you are back to flaky tests looking like passes.

Sharded runs: one JUnit file

With sharding, each machine can write its own JUnit file, and many CI systems accept several files (Azure's PublishTestResults@2 with mergeTestResults: true is the example in Playwright's docs). For a single file, run the blob reporter on each shard and produce JUnit once with merge-reports, giving it an output file in a merge config or through PLAYWRIGHT_JUNIT_OUTPUT_NAME. The merge-reports guide shows both.

What JUnit cannot carry

A JUnit file describes one run. Even with includeRetries, it cannot say whether a failing test also fails on your main branch, whether it was flaky last week, or which tests should have run but never reported because a shard crashed. A missing test is simply absent from the XML.

TurnSignal keeps every attempt of every test across runs and labels each failure new, also failing on main or known flaky, with tests a crashed shard never reported listed as Missing. It runs as one more reporter next to junit, so your CI keeps its JUnit integration. More on choosing reporters in Playwright test reporting in CI; setup is in the docs.