# Playwright tests missing from CI results: crashed shards, OOM kills and timeouts

> Why a sharded Playwright run can report fewer tests than it has, how to spot the gap, and how to keep results when a CI machine crashes or runs out of memory.

Published 2026-09-28, updated 2026-09-28. Canonical: https://turnsignal.ai/blog/playwright-missing-tests-crashed-shard

## In short

- When a CI machine is killed (out of memory, job timeout, lost runner), the Playwright process on it never finishes, so it writes no report. Its tests silently disappear from the merged results.
- A green merged report can therefore hide tests that never ran. Compare the number of tests you expected with the number that reported.
- Tests stopped by `globalTimeout`, `--max-failures` or Ctrl+C are different: Playwright is still alive and marks them as interrupted or "did not run".
- To keep results from a machine that dies, write each result to disk as it happens and send it from a step that always runs.

## The symptom: fewer tests than you have

You split a suite across four machines with `--shard=1/4` … `--shard=4/4`, merge the reports, and the result is green. But the suite has 1,200 tests and the report lists 900. Nobody notices, because a merged report shows what it received, not what it should have received.

This happens more as suites grow. More machines means more chances that one of them runs out of memory, hits the job time limit or loses its runner, and the bigger the shard, the more tests vanish with it.

## Why a killed machine leaves no report

Playwright's reporters produce their final output at the end of a run. With sharding, the usual setup is the `blob` reporter on each machine and `npx playwright merge-reports` in a final job. A blob report is written when the run ends; if the whole machine or the Playwright process is killed first, there is no blob for that shard, and `merge-reports` merges the shards that did finish.

```ts
// playwright.config.ts
export default defineConfig({
  reporter: process.env.CI ? 'blob' : 'html',
});
```

The common ways a machine dies before Playwright can finish:

- **Out of memory.** Browsers are memory-hungry. When the machine runs out, the operating system kills a process, sometimes a browser, sometimes Playwright itself, or the CI provider stops the job.
- **The CI job time limit.** On GitHub Actions a job is cancelled after `timeout-minutes` (360 minutes unless you set it). A hanging test can eat the whole budget.
- **Lost or preempted runners.** Spot instances, autoscaled Kubernetes pods and self-hosted machines can disappear mid-run.
- **A shell `timeout` or a killed container.** Anything that sends SIGKILL gives Playwright no chance to write a report.

> A single crashed worker is not the same thing. If one worker process dies, Playwright reports the test it was running as failed and continues with a fresh worker. It is the death of the whole Playwright process, or the machine, that loses results.

## Stopped on purpose: interrupted and not run

Some stops are handled by Playwright, so they do show up in the report. With `globalTimeout` (a limit for the whole run) the run ends with status `timedout`. With `--max-failures` (or `maxFailures` in the config), Playwright stops after that many failures. Ctrl+C gives status `interrupted`. In each case the run itself says why it stopped, tests cut off mid-run get the `interrupted` status, and tests that never started are listed as "did not run".

That is useful: the report is honest about what happened. It is also a reason to read the totals and not only the colour, because a run stopped early can have no failures at all among the tests it did finish.

## How to spot the gap yourself

1. List what should run: `npx playwright test --list` prints every test (and the total) without running anything. Run it with the same `--project` and `--grep` options as CI.
2. Count what reported: the merged report shows the number of tests it contains.
3. Compare the two on every run, and fail loudly (or at least warn) when they differ. A small script in the merge job is enough.
4. Record memory: log the machine's free memory during the run so an out-of-memory kill is not a mystery afterwards.

## Keep results when a machine dies

The fix is to stop depending on the end of the run. A custom reporter receives `onTestEnd` for every test as it finishes; if it appends each result to a file right away, everything up to the moment of the crash is on disk. A later CI step with `if: always()` can then send it, as long as the machine or workspace survived. When the whole machine is gone, you still know which tests should have run, so you can list the ones that never reported.

Two settings reduce the damage in the first place: set `timeout-minutes` on the job a little above your normal run time, so a hang ends early instead of after six hours, and lower `--workers` on small runners, which is the most common fix for out-of-memory kills.

## How TurnSignal handles it

[TurnSignal](https://turnsignal.ai/)'s reporter writes every result to disk in CI before sending it, and at the start of each shard it records the list of tests that shard is going to run. When a shard ends or goes quiet, any planned test that never reported is shown as **Missing**, with the likely reason (for example, the machine stopped responding) and the lowest free memory seen on that machine. `npx turnsignal upload` in an always-run step (or the TurnSignal GitHub Action) sends whatever was left on disk, and running it twice is safe. For containers and pods, `npx turnsignal run -- npx playwright test` uploads before the container exits.

The reporter never changes your exit code, and it works next to your existing reporters, including `blob` and `html`. See the [docs](https://turnsignal.ai/docs) for setup.

## FAQ

**Why does my merged Playwright report have fewer tests than my suite?**

Usually one shard was killed before it finished (out of memory, job timeout or a lost runner), so it never wrote its blob report. `merge-reports` merges only the shards that finished.

**How do I know how many tests should run?**

Run `npx playwright test --list` with the same options as CI. It prints every test and the total without running them.

**What does interrupted mean in Playwright?**

The test was running when the run was stopped, for example by `globalTimeout`, `--max-failures` or Ctrl+C. Tests that had not started are listed as "did not run".

## Sources

- [Playwright docs: Sharding and merging reports](https://playwright.dev/docs/test-sharding)
- [Playwright docs: Reporters (blob)](https://playwright.dev/docs/test-reporters)
- [Playwright docs: TestConfig.globalTimeout and maxFailures](https://playwright.dev/docs/api/class-testconfig)
- [Playwright docs: Reporter API (onTestEnd, FullResult.status)](https://playwright.dev/docs/api/class-reporter)
- [Playwright docs: Command line (--list, --max-failures, --workers)](https://playwright.dev/docs/test-cli)
- [GitHub docs: jobs.<job_id>.timeout-minutes](https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax#jobsjob_idtimeout-minutes)