Playwright AI Revolution in Test Automation

The modern IT industry is undergoing a radical transformation. Teams face the complex task of creating dynamic, scalable, and secure digital solutions. Furthermore, achieving high software quality should not affect delivery speed.

Achieving these goals is impossible without a well-structured testing strategy, an important part of which is test automation. Moreover, the reality of today is such that automating tests is more optimal with the power of AI.

Artificial intelligence and generative AI may be the most important technology of any lifetime,

says SalesForce CEO Marc Benioff

— How are teams using AI testing technology in their workflow when using Playwright 🤔

⬇️ Let’s explore further!

Codegen Playwright

Codegen — is a powerful built-in Playwright tool that automatically creates test cases by recording user interactions with a web application or website. Its usage like an AI tool is the most predictable. Playwright Codegen is not only fast, but also quite a smart testing tool 😃

To get started with Playwright Codegen, you should run the next command in the terminal:

npx playwright codegen

This will open two windows: a browser, where you will perform various actions, and the Playwright Inspector. The latter will display the Codegen window filled with the code generated as a result of your interactions. It will look like this:

All these actions take just a few seconds. This speeds up the execution of testing tasks and allows automation engineers to implement best testing practices in their projects.

♨️ Anyway, it is important to remember — Playwright Codegen is not perfect, and all the code it generates should be double-checked for alignment with the team’s needs.

AI testing & Automation with ChatGPT

An alternative to classic test automation with Playwright Codegen is generating test code with AI. To do this, in a dialogue window – for example, the well-known is a GPT chat – plain text instructions should be entered with its interface. They should contain information about what exactly needs to be tested, which device configuration to use, how to write a particular piece of code or improove it etc.

Of course, the generated by AI Playwright tests may contain some inaccuracies in your testing scripts. For example, your AI code may use locators that are not recommended by the framework. These include page.click() and page.fill() Using the same plain text commands, you can point out the inaccuracies to GPT and ask it to correct them to the proper version – in our case, role-based locators.

To avoid constant revisions, it is advisable to include preliminary plain text prompts in the request to generate the Playwright AI tests. A small recommendation – example prompts can be found on cursor.directory:

Thus, to successfully generate Playwright tests using AI, you should:

✅ use a specialized prompt to help the tool understand what it is dealing with;
✅ verify the accuracy and relevance of the generated code;
✅ provide as much context as possible (e.g., JavaScript and HTML files – anything that supports your app).

Given that modern web applications have highly complex architectures, fulfilling all these requirements can be challenging. Therefore, Playwright Codegen remains a more convenient option for the rapid creation of tests.

GitHub Copilot for Playwright Code Generation

Copilot AI is a powerful tool that analyzes the context of your code and then suggests complete Playwright test scripts that you can use for test creation, debugging, refactoring and maintenance of your code. It is seamless with all Microsoft Products.

For instance, Copilot identifies redundant lines, incorrect selectors, unused variables, or better Playwright-specific API methods to enhance maintainability and performance. Copilot AI may be trained on public code or your own. Moreover, Copilot’s alternative AI code can simplify intricate test scenarios, fix asynchronous issues or propose ways to handle flaky tests, saving time spent thinking up the best solutions at times.

Testomatio AI test management for your workflow

Recently, testomat.io team developed a couple of new AI features from testomat.io, including AI-powered test case creation with advanced analytics for QA test managers. These improvements provide smarter test management, increased productivity, and full project control.

🧠 It suggests optimized tests with AI for popular test automation frameworks like Cypress, Jest, Cucumber, PyTest, and JUnit etc, including the Playwright framework. AI feature automatically generates test case ideas based on the content of the test suite. Users can review the suggested tests, select the relevant ones and add them directly to the suite or reject them and re-generate the new ones. This accelerates test case creation, enhances test coverage by identifying overlooked scenarios, and reduces manual effort while maintaining high testing quality.

Why Testomatio AI Test Management insights worth implementing?

• Increased productivity. Save time with faster synchronization of your manual and automated test sets.
  Improved test quality. AI Flaky test detection identifies patterns in test execution to pinpoint tests that pass or fail inconsistently.
• Intelligent test maintenance. Maintaining test cases in dynamic environments can be tedious. Testomat.io’s AI identifies duplicates, obsolete or redundant test cases or test steps based on them. Test history trends over time and Analytics may empower teams about gaps in test coverage after updates.
• Faster Releases. Predictable analytics and smart prioritization help identify and address high-risk areas in test strategy promptly.
• Seamless Collaboration. Enable non-technical stakeholders to contribute using plain language test creation.

We provide a Custom AI Provider integration. It enables companies to use their preferred AI provider, such as OpenAI or Anthropic, for all AI-driven features. This feature offers flexibility in leveraging AI capabilities while ensuring compliance with data-sharing policies and regulations.

Also, as an alternative, you can use third-party AI-based tools designed to optimize work with the popular framework.

Other Third-Party AI Tools work with Playwright

Playwright itself does not provide users with artificial intelligence-based functionality. As a result, its enthusiasts have to turn to third-party tools to optimize web automation with their favourite library.

QA teams who find Codegen’s capabilities insufficient may consider AI-based automation testing tools like Auto Playwright or ZeroStep. Below you can see an example of a piece of code generated by ZeroStep AI-based tool.

Now, take a look at their features and advantages 👀

ZeroStep

ZeroStep – is one of the most effective testing solutions that leverages the power of GPT-3.5 and GPT-4 for the rapid automation of Playwright tests.

Using the platform involves writing instructions in natural language. Simply tell the tool how you need to interact with the website, and AI will handle the rest for you!

To get started with ZeroStep, use the command:

npm i @zerostep/playwright -D

ZeroStep Playwright AI features for optimizing web testing

• Test development without writing code. All the user needs to do is interact with the application via the UI. The tool will generate the tests automatically.
• Cross-browser and cross-platform testing. The platform supports testing in all popular browsers (e.g., Chrome, Firefox, Safari) and on various platforms (Windows, macOS, Android). This guarantees that the digital solution performs consistently across various environments.
• Self-healing tests. The tool automatically adapts tests to minor changes in the product, minimizing the need for manual intervention.
• Seamless integration with third-party platforms. ZeroStep easily integrates with CI\CD solutions such as GitHub Actions, GitLab, Jenkins, etc.
• Detailed reporting and analytics. The platform provides users with comprehensive reports featuring visualized results. Additionally, information on test coverage, performance, and other analytics is available.

Advantages of ZeroStep AI

• Affordable pricing. Users get access to 500 free AI() function calls per month. If you need to increase the limit, the cost starts at $20.
• Optimized collaboration. The no-code approach allows non-technical team members to participate in testing.
• Increased accuracy in software testing. Smart test maintenance adapts tests to changes in the application. This helps keep them up-to-date and prevents inconsistent test results.
• Fast Feedback. Real-time test execution and integration with CI\CD pipelines enable teams to respond to issues promptly.

Auto Playwright

This is another tool that allows teams to integrate artificial intelligence into their Playwright workflows. It serves as an intelligent intermediary between the user’s plain text instructions and the test code.

This approach ensures high-quality software products in today’s dynamic digital landscape.

⚙️ Setting up and configuring Auto Playwright is straightforward. To get started, follow these steps:

1. Install the dependency using the command:

   npm install auto-playwright -D

2. Create an OpenAI API key (you will need to register on the OpenAI platform for this).
3. Export the OpenAI API token by running the command:

   export OPENAI_API_KEY='sk-...'

4. Configure the environment variables following the guidelines provided in the official documentation for the tool.

After completing these steps, you can use the Auto Function to create tests.

Let’s look at a practical example of code generated by Auto Playwright. Suppose we need to verify the correctness of the user registration process in an application.

To generate an automated test case, compose instructions containing the following directives for the tool:

→ navigate to the registration page;
→ fill in the user details;
→ submit the registration form;
→ verify the results of checks – a success message should be displayed upon registration.

The generated Auto Playwright AI test will look like this:

import { test, expect } from "@playwright/test";

import { auto } from "auto-playwright";



test('User registration', async ({ page }) => {

 await auto('go to the registration page', { page });

 await auto("fill in name, email, and password", { page, test });

 await auto('submit the registration form', { page });

 const registrationSuccess = await auto('is the registration successful?', { 

    page, 

    message: 'Registration successful' 

  });



expect(registrationSuccess).toBe(true);

});

Learn more about the tool by watching the Execute Automation Academy by Karthik K.K video: Auto-Playwright – Run Playwright Test with AI 🚀 (Using GPT 3, GPT 4 Turbo)

Key features of Auto Playwright:

• Automatic waiting. The tool automatically waits for elements to be ready for interaction, reducing the number of flaky tests.
• Cross-browser testing support. You can run tests in Chromium, Firefox, and WebKit to ensure your solution works consistently across different browsers.
• Code reduction. Auto Playwright streamlines page interactions and reduces repetitive code. For instance, interacting with a registration form field only takes a single line of code instead of multiple lines. To demonstrate, let’s refer to the test from the previous example:

 await auto("fill in name, email, and password", { page, test });

• Automatic element visibility checks. You do not need to write specific assertions for visibility, reducing the complexity of test cases.
• Retry on failures. The tool retries test executions in case of common errors (e.g., browser load failure, element timeout, etc.). This makes tests more reliable and stable.
• Easy navigation. Auto Playwright supports navigation between different pages, tabs, or browsers.

Advantages of Auto Playwright:

• Cost savings. Auto Playwright is an open-source framework, meaning you can use this effective testing solution completely free of charge.
• Improved collaboration on the project. With this Playwright AI tool, you can create readable, concise autotests. This simplifies the work for less experienced testers and allows non-technical specialists to participate in testing.
• Reduced risk of errors. This is facilitated by the advanced features of the Auto Playwright AI tool, including auto-waiting, element visibility checks, retries, and more.
• Wide integration capabilities with testing frameworks. You can leverage Jest, Mocha, Jasmine, or other platforms while benefiting from the power of AI with Auto Playwright.

Comparison of ZeroStep & Auto Playwright

To help you choose the optimal tool for your project, here’s a comparison of ZeroStep and Auto Playwright in a clear tabular format.

What Are the Limitations of Tools for Playwright AI Testing?

In this article, we’ve provided plenty of evidence that artificial intelligence is the future of test automation. However, it’s crucial to consider the limitations that might emerge when using AI-based testing tools. Let’s take a look at the potential challenges teams may face when deciding to implement AI in their projects.

Limitations of ZeroStep:

• Limited User Capabilities. The tool uses a codeless approach to writing test cases. As a result, it may be difficult to automate complex and non-standard workflows.
• Scalability Challenges. The tool is primarily designed for small and medium-sized projects, and managing large test sets can present challenges.
• Basic Integration Capabilities. Users may experience difficulties integrating ZeroStep into their workflows.

Limitations of Auto Playwright:

• Steep learning curve and need for coding skills. This is a code-based tool that requires knowledge of JavaScript or TypeScript. It will also take time to learn if you’re not familiar with Playwright.
• Manual setup. Before starting testing, users must manually configure the testing environment. This can be a complex task.
• Limited test management capabilities. To manage test cases and generate advanced reports, integration with third-party tools is required.

Both tools are powerful solutions for web application test automation. They allow teams to adapt to the dynamic nature of web development and demonstrate excellent results. When choosing a platform, consider the limitations of each. This will help meet your specific testing needs.

Bottom Line

Playwright with AI emerged in response to the need for reliable and efficient web application testing by QA teams. This approach combines the reliable functionality of the Playwright automation library with the advanced capabilities of artificial intelligence.

Testers interested in the rapid creation of tests can use the built-in Playwright Codegen, optimizing its capabilities with ChatGPT and other similar systems. Additionally, third-party AI-based tools are at their disposal.

— Which option to choose 🤷 We hope our article has helped you make a decision. If you still have questions, reach out to a testomat.io manager for expert consultation on any aspect of testing.