On my journey to select the best tool for our end-to-end tests for our teams, I tested out the recently announced Playwright tool from Microsoft. This tool is the new version of Puppeteer (as the core team members moved to Microsoft).
Info: The benefits of E2E tests are that it validates if your features work (or keep working when implementing changes), and also provides a more code-driven approach of documenting the features and user flows.
Like Puppeteer, Playwright allows you to automate browser tasks (navigating to a page, taking a screenshot, interacting with the page, and more). The best functionality is that it has cross-browser support. You can run tests against Chromium, Firefox, and WebKit.
Important: Playwright is still in development, so do not expect everything to work.
It is also good to know that Playwright is not a testing tool, but you can use it for this functionality. That is where Jest is coming into play. You can also use other test runners, but in my experience, Jest is the one that works with minimal configuration.
Info: As a reference, you can use the Puppeteer article, which I wrote some time ago: Testing the UI of your SPFx solution with Puppeteer and Jest.
Project: The sample project for this article can be found on GitHub - https://github.com/estruyf/playwright-jest-e2e.
Things you need to start testing
To start, you will only need Playwright and Jest for your project dependencies:
| |
Once you have created a new project and installed these two dependencies, all you need is the configuration to run the tests.
As I said, Jest requires minimal configuration, so you only have to update the test command in your package.json file to execute: jest.
Writing your first test
Your first test could look like this:
As you configured to run Jest as the test for npm, you are now able to run it via npm test.
Adding cross-browser support to your test
Until now, it is not so different compared to Puppeteer. I only had to change a small number of methods to migrate my Puppeteer tests to Playwright. The coolest part about Playwright is cross-browser support. You can configure this in your tests by adding a simple for loop.
The previous test will look like this:
Run npm test again, and Jest will now automatically test against all supported browsers.
Adding support for SharePoint Online testing
Testing against SharePoint is the same as how it was for Puppeteer. To authenticate, you can use the node-sp-auth dependency. With this dependency, you can get the authentication headers, and pass that to the Playwright browser.
You can find the code for the SharePoint Authentication can here: SharePoint Authentication helper.
Running on Azure DevOps
Running it on your local machine is useful for verifying when you are extending/changing features and want to know if everything works as expected. The real value lies in the automation part.
Azure DevOps is great, and Playwright can just run on the available hosted build agents without the need for any special configuration.
In the sample I created, you can verify the provided pipeline: azure-pipelines.yml - e2e.testing.yml template.
The whole test pipeline looks like this:
Automatically screenshots are taken before and after each test. The pipeline publishes these screenshots as artifacts.
Important: The sample tests are just as a reference for you to get started.
After all tests, the JUnit file gets published, and you can verify the test outcome. You can find the configuration for JUnit in the jest.config.js file.
If you want to run a similar pipeline on your environment, you need a variable group with the following settings:
Info: The sample project for this article can be found on GitHub - https://github.com/estruyf/playwright-jest-e2e.
Note: Another approach might be to create a stage for each browser (and pass in the browser name as a parameter) and run the tests in parallel. This will speed up the tests.
Conclusion
If you know Puppeteer, or used Puppeteer already. Playwright will be very easy to get you started. There are not a lot of differences.
I like the simplicity of Playwright and how easy it is to use it with a testing framework like Jest. For complete test runs, it is great, but when it comes to single test runs, it requires a bit more manual steps for your testers or developers.
I also miss a video option like in Cypress. This makes it easy for testers/developers to understand what might have gone wrong during the automated tests. Screenshots are good, but videos make it a lot easier.
The most significant advantage is that the tests run in a “real” browser. Not in an iframe like Cypress. Maybe that does not matter for most solutions you are testing, but for SharePoint, this makes a huge difference. When a SharePoint page gets loaded in an iframe, you do not have all SharePoint page controls available.