Playwright and GitHub Actions

Today we are looking at integration with CI/CD pipelines, as that is, let's be honest, where you get real "bang for your buck" when it comes to automation. As a bonus, we are also using GitHub Actions for the first time, so a double whammy of learning that too!

So, let us begin. A bit of a search gave us the basic config required to run the test scripts in GitHub. So our .github/workflows/playwright.yml file looks like this:

name: Playwright Tests
on:
  push:
    branches: [ main, master ]
  pull_request:
    branches: [ main, master ]
jobs:
  test:
    timeout-minutes: 60
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v3
    - uses: actions/setup-node@v3
      with:
        node-version: 18
    - name: Install dependencies
      run: npm install -g yarn && yarn
    - name: Install Playwright Browsers
      run: yarn playwright install --with-deps
    - name: Run Playwright tests
      run: yarn test
    - uses: actions/upload-artifact@v3
      if: always()
      with:
        name: playwright-report
        path: playwright-report/
        retention-days: 30

What this says in English is:

For every Push and Pull Request, spin up an Ubuntu VM using the latest version

Install the code dependencies

Install the Browsers to Run On

Run the Tests

Upload the HTML report

Pretty simple! Fortunately, none of the code so far had been pushed, so that is what we did. And...

Our first test failed! There is however no need to fear, as this was expected.

Looking at the published report, which we have had to download and view via playwright show-report :

It is the tests from our first test, where the search page didn't work via Playwright:

What was pleasing is that the tests have been run across a range of browsers by default, which is a nice touch.

With that in mind, our next step is to increase the number of browsers to be tested. This is achieved fairly simply with a change to the playwright.config.ts file. We are adding the following. to add mobile and branded browsers (basically Google Chrome as opposed to a Chromium emulator):

/* Test against mobile viewports. */ { name: 'Mobile Chrome', use: { ...devices['Pixel 5'] }, }, /* Test against branded browsers. */ { name: 'Microsoft Edge', use: { ...devices['Desktop Edge'], channel: 'msedge' }, }, { name: 'Google Chrome', use: { ...devices['Desktop Chrome'], channel: 'chrome' }, }, ],

So let us see the results when we push these changes:

As expected, there are failures. Let us look at the detailed report to see what it was:

We have one extra failure were weren't expecting. The navigation test on Mobile Chrome:

Looks like the "Writing Tests" link is to blame. Diving into the trace, it looks like the link has "Writing tests >>".

So, what have we discovered? Well, integration with CI/CD pipelines is pretty easy, and the reporting is pretty good. I would say Cypress slightly has the edge here, as the videos and screenshots can be uploaded as artefacts and accessed easily. But other than that, they are about on a par.

And GitHub Actions? My new CI/CD friend. So easy to use and also just there with your repo. If we can get it integrated with a test or workflow tool (which should be really easy) I think we have a winner!