Create GitHub issues on failed Playwright tests

As various projects I maintain for customers include end-to-end (E2E) tests using Playwright, which run on a daily schedule and on every new release, I wanted to automate following up on failed tests. Instead of manually creating issues, I automated the process using GitHub Actions and the GitHub API. This worked great, but I thought to myself, why not share this with the world? So, I created a GitHub Action that does exactly that.

Show image

In this blog post, I will explain how you can use GitHub Action to create issues on failed Playwright tests automatically.

The GitHub Action

The GitHub Action is called Playwright Issue Creator and is available on the GitHub Marketplace.

The Playwright Issue Creator action relies on the JSON report created by the JSON reporter to verify the test results.

Usage

Include the JSON reporter to your Playwright config

As mentioned, the GitHub Action relies on the JSON report created by the JSON reporter.

You can include the JSON reporter in your Playwright config or use the --reporter json flag when running your tests.

Using the JSON reporter in your Playwright config

1
import { defineConfig } from "@playwright/test";
2

3
export default defineConfig({
4
  reporter: [
5
    ["json", { outputFile: "results.json" }]
6
  ]
7
});

Using the --reporter json flag

1
PLAYWRIGHT_JSON_OUTPUT_NAME=results.json npx playwright test --reporter=json

Create a GitHub Action workflow

Create a new GitHub Action workflow in your repository and add the following content:

1
name: Playwright Tests
2

3
on:
4
  schedule:
5
    - cron: "0 6 * * *"
6

7
jobs:
8
  test:
9
    timeout-minutes: 30
10
    runs-on: ubuntu-latest
11

12
    # Assign write permissions for issues to the GITHUB_TOKEN
13
    permissions:
14
      contents: read
15
      issues: write
16

17
    steps:
18
      - uses: actions/checkout@v4
19

20
      - uses: actions/setup-node@v4
21
        with:
22
          node-version: lts/*
23
          cache: npm
24

25
      - name: Install dependencies
26
        run: npm ci
27

28
      - name: Run Playwright tests
29
        run: npx playwright test
30

31
      - name: Playwright issue creator
32
        if: always()
33
        uses: estruyf/[email protected]
34
        with:
35
          report-path: results.json
36
          issue-prefix: "E2E: "
37
          issue-labels: "playwright, bug"
38
          add-project-label: true
39
          add-comment: true
40
          job-summary: true

Run your tests

After you set up the above GitHub Action workflow, the test will run daily at 6 o’clock. You can, of course, add more triggers or run the tests on every push.

Happy testing!