Playwright Testing Tutorial on How to Organize an Advanced (Scaled) (e2e/unit) Testing Framework

One of the main trends in modern software development is the rapid release of ready-to-use digital solutions of premium quality on the market. Accelerating the testing process by automating it is one way to achieve this goal. At testomat.io, we constantly improve our system for managing automated tests and pay maximum attention to extending its integration capabilities. Look at how it implements the option to collaborate with one of the testing frameworks, Playwright.

What Is Playwright?

Playwright framework is a Node.js library that enables end-to-end automated web testing.

It supports cross-platform testing and provides the ability to use a single API to work with browsers based on Chromium, Firefox, and Webkit. It also, by default, launches headless browsers that don’t show UI and are available via the command line. Also, Playwright supports automation of full (not headless) Microsoft Edge based on Chromium.

Microsoft created the framework in 2020, is actively supported by the company, and is open source. It is very fast, even in the case of large-scale projects, and is rapidly gaining popularity. You can find comprehensive information about working with Playwright on the Microsoft website in the Documentation section or on GitHub.

Automation Testing With Playwright

Playwright is a flexible test automation environment with a clear syntax that supports test execution across domains and multiple browser pages. Other features of the Playwright testing tool include:

• Support for cross-browser web automation. You can perform all the tests in Chrome, Firefox, Edge, Safari, and Opera.
• Ability to perform cross-platform testing. Playwright framework operates with Windows, Linux, and macOS.
• Multi-language. Test software regardless of the programming language you use – several languages are available to you, including JavaScript, TypeScript, Python, Java, and .NET.
• Providing reliable end-to-end testing of modern web apps thanks to the automatic wait option. This means that Playwright waits for items to be ready before launching actions, thereby saving you from the problem of flaky tests.
• Running tests in an isolated environment called browser contexts. This approach saves time because creating a new browser context is much faster than launching a new one.
• The ability to interact with a multipage site as a real user. This is facilitated by modern Playwright architecture.

As you can see, the Playwright testing framework is a great alternative to other testing tools that have long gained credibility in the market. It’s an ideal solution for end-to-end, functional, and API testing.

Why Do Many QA Teams Choose Playwright?

Playwright is a relatively new framework that was created with the needs of modern Agile teams in mind. We tried to figure out the reason for its rapidly growing popularity and highlighted its main advantages.

Working with the Playwright testing tool, you will be able to:

1. Run tests with a single click thanks to the VS Code extension (works with Playwright v1.19+).
2. Debug a Playwright test script using trace viewers console logs, Playwright Inspector, VSCode Debugger, and Browser Developer Tools.
3. Perform parallel test execution locally in the Selenium Grid or split the test suite for running on different machines.
4. Create an HTML report about the test results. The following test artifacts are available for visualization of test progress data in this framework: screenshot, test execution screencast, error log, and execution trace.
5. Use the built-in reports (JSON, JUnit, HTML) or create custom reporters.
6. Record user actions as test code using Playwright’s codegen tool.
7. Run groups of tests that can be pre-linked thanks to the test marker feature (e.g., ‘@duration=long’).
8. Set up Continuous Integration simply and quickly, thanks to the docker images provided.

Test management allows you to import Playwright tests into your test management system in a few clicks and test software even more efficiently.

Below we’ll describe how to work with Playwright launches in our TMS:

Our test management solution provides seamless integration with Playwright, which allows you to import tests into your test management system in a few clicks and test software even more efficiently.

Below we’ll describe how to work with Playwright launches

Getting Started Playwright From Example (ready boilerplate)

On our website, you will find documentation with a detailed description of TMS, its integration capabilities with popular automation frameworks (Cucumber, Protractor, TestCafe, etc.), as well as the Jira project management system. For now, let’s focus on the Playwright tutorial and the actions required to run the Playwright test in test management.

Our TMS supports Playwright in the next combination:

• Playwright+BDD(Gherkin)
• Playwright+JavaScript
• Playwright+TypeScript

Steps you need to follow to run the example on testomat.io test management:

Note: Before starting, be sure to meet all the conditions that are necessary for the correct operation of the system.

Prerequisites For Playwright

Playwright requires installation of Node.js version 12 or higher. To check if your platform version is compatible, run the node -v command.

After that, install Node.js or upgrade it by running npm install –save-dev. This manipulation will install the Node package into the node_modules/ folder of your project. If all went well, you can move on to testing. Where should we start?

#1 Create a new test project

Register or log in to your test management account. After that, you will have the opportunity to create a new test project – to do this, click the Create button in the upper right corner of the opened window. Select the type of the project and enter its name.

#2 Import automated tests

In a new test project, you will be able to import automated tests by selecting the appropriate menu item in the upper right corner of the window. TMS allows you to do this in a few clicks:

In the Import Project from Source Code window, select the desired framework (Playwright), the programming language used (JavaScript/TypeScript)

After then choose the desired operating system (Mac/Linux/Windows) and copy the generated command with the API key and execute it in CMD. This will start importing tests from the source code into the test management system.

Look at the console, test management importer in this process analyzes code accurately and displays the number of founded tests in the test framework

Go back to your project and look at the number of tests found. If necessary, you will be able to click on the desired test to view the steps it contains.

In such a way, test management also allows you to track the relevance of imported tests without any extra effort. TMS will record every change made and warn you if the current version differs from the one imported from the source code; such tests will be assigned the Out of Sync status. This will allow you to make a timely decision to cancel incorrect changes.

Import autotests and create manual tests across (optional)

In our TMS, you can not only import autotests from the source code but also create manual tests. To do this, after creating a new project, select the Create Suite menu item and then Add new test.

Now you can run manual test cases. Mark them as passed or failed, and add additional information (in text format, screenshots, etc.). As a result, you will be able to get summary reports of manual and automated test results in one place.

Organize your Playwright tests in the best way

When a project has a large number of tests, the ability to structure them with tags and a tagging system becomes relevant.

You also have the ability to sort, reorder, filter, and find the right test case. More detailed about the range of possibilities of how to speed up QA efficiency by following the link.

#3 How Do I Create A Report In Playwright?

To get a detailed report of the test results, you should first install testomat.io Reporter in the repository. To do this, copy and paste the following command into CMD:

npm i --save-dev @testomatio/reporter

#4 Configure .Conf File:

We suggest adding our reporter (test management plugin) along with existing reporters in wdio.conf.js. For this purpose copy configuration suggested by TMS to the project which can find out in the DEMO project as well.

reporter: [
  // ...
  ['./node_modules/@testomatio/reporter/lib/adapter/playwright.js', {
    apiKey: process.env.TESTOMATIO,
  }]
],
        

#5 Execute Playwright Scripts

TESTOMATIO='API' npx playwright test

As a result, you will have access to a real-time status report before the test run is completed. If something goes wrong, you can view the execution trace, analyze test scenarios, and examine the attachments to clearly understand the problem.

Once testing is complete, a link to the full report will be generated, which can be shared with stakeholders as needed.

We can evaluate the results of Playwright testing with deep analytics; the corresponding section is available on the test management dashboard.

You can use it to track automation coverage, and the number of ever-failing, slowest, flaky, and never-run tests. It also provides information about the test environments used and test coverage by tags. The following testing metrics are available on the Project Analytics dashboard and Global Analytics dashboard. Our team are currently working on expanding with new widgets.

Running Playwright Test On CI\CD

CI\CD is a leading development practice that aims to speed up the release of the finished product to the market. We at testomat.io support this trend and have implemented in our product seamless integration with popular CI\CD tools, including GitHub, GitLab, Jenkins, Bamboo, and CircleCI.

In addition to a quick software release, running Playwright testing on CI\CD allows:

• cooperate more closely with all members of the Agile team
• reduce risks through the ability to control the elimination of defects
• save budget by eliminating bugs at an early stage
• get a premium quality product

Comparison table: Playwright VS Cypress VS Selenium