For more articles like this, visit our Product Insight corner
If your job involves using automation testing frameworks and tools such as Playwright, then we would love to hear from you. From now till Apr 21, 2023, complete our survey and you will get a chance to win exclusive gifts from us!
For automation engineers and QA practitioners, Selenium has been the go-to choice for its versatility and vast community support. However, the automation community is now embracing other promising non-Selenium tools, such as Playwright: a powerful, open-source, feature-packed framework built by Microsoft.
Since its introduction in 2020, Playwright has steadily gained popularity. As of March 2023, some interesting statistics from the GitHub repository of Playwright include:
- 48.3k stars
- 2.4k forks
- 382 contributors
So why has Playwright become so popular? How is it catching up with other widely used frameworks like Selenium and Cypress?
This article aims to answer those questions by discussing the capabilities and limitations of Playwright, as well as the potential for future adoption of this framework into the Katalon Platform.
Playwright is an open-source framework designed for reliable end-to-end testing of modern web apps. As a cross-browser, multi-platform tool, it supports all popular rendering engines, including Chromium, Firefox, and WebKit, on Windows, macOS, and Linux.
Despite being a relatively new entrant to the automation market, Playwright has gained significant traction. In the 2022 State of JS developer survey, Playwright showed impressive growth in usage and retention rates.
Let’s dive into the feature set of Playwright.
Compared with other existing solutions, Playwright has successfully accommodated the features of a complete web testing framework. Some of the prominent features include:
- Quick and easy setup: Playwright requires very minimal installation to get started.
- Cross-browser support: The framework supports all the core engines of popular browsers: Chromium for Chrome and Edge, WebKit for Safari, and Firefox.
- Auto-wait: A form of smart wait feature. Playwright waits for dynamically loaded elements before taking the next steps, as opposed to using the flaky fixed wait time.
- Web-first assertions: Playwright automatically retries checks until a necessary condition is met, before proceeding.
- Execution trace: The tool can trace test execution and provide necessary insights in the form of videos and screenshots.
- Browser context: Instead of using multiple browser sessions, Playwright leverages the use of browser context to create multiple browser profiles for parallel testing. This also delivers complete test isolation with minimal overhead.
In addition to the rich feature set, the team at Microsoft has also equipped Playwright with powerful tooling.
Test Generator and Inspector: Playwright comes with the ability to generate tests out-of-the-box, making it a great way to quickly get started with testing. When you launch the Test Generator, Playwright opens two windows: a browser where you can interact with the website under test and the Inspector, which helps you write and debug scripts.
Trace Viewer: Being able to trace test execution and record snapshots means that you can perform time travel debugging with Playwright. The Trace Viewer tool allows you to step back and forth through each action of your tests and visually see what was happening during each action.
Multiple Reporters: Playwright comes with multiple built-in reporters for different needs, such as HTML, JSON, JUnit, and the ability to create custom reporters.
To quickly view a test report, you can serve up an HTML reporter on your local machine and easily walk through the steps of your test. Combining this with the trace file, Playwright gives you powerful options for dynamically viewing reports and, at the same time, debugging your tests.
While Playwright is considered on par with top testing frameworks, it does come with limitations, such as:
Limited community and documentation support: Although the interest for Playwright is growing, it’s still relatively small. When you hit a roadblock with the tool, it’d be more difficult to find documentation and recipes.
Lack of infrastructure for collaboration: Playwright is a feature-packed framework but does not provide the infrastructure to run tests and collaborate with other team members. This can make it challenging to share assets, get updates, manage test reports, especially when dealing with frequent code changes.
The current practice for Playwright users is to work with Visual Studio Code, and synchronize with team members through Git. In a large test project with many contributors, this can be challenging as version control only supports managing source code, not test artifacts. When reviewing changes, testers can be bombarded with irrelevant data and not entirely invested in the testing activities.
To run Playwright tests, users have to prepare test environments and schedule test executions, which can be difficult to keep track of. Also, test reports from these environments need to be centralized. The default Playwright report files can be handy, but they provide little insights for cross-platform testing.
While Playwright is a powerful automation tool, it’s hard to complete the testing pipeline using it alone. In an enterprise setting, the pipeline involves multiple stakeholders, including developers, quality engineers, QA managers, and others who want to audit data and ensure standards are met.
For this reason, test managers and team leads using Playwright often look for a comprehensive testing platform that allows them to complete their pipeline for automation, from test planning, authoring, management, and execution to reporting and analytics. This is where the Katalon comes in.
The addition of Playwright into Katalon, our all-in-one platform, can open up new possibilities for testers. Let’s imagine what the Playwright-Katalon integration would look like.
By setting up a Katalon-Playwright test project, users can create a central container for all testing activities and synchronize it with other team members. The multi-level management implemented in the Katalon Platform, including Account, Organization, Team, and Project with different roles, facilitates the involvement of stakeholders in the enterprise testing pipeline. For example, Billing managers can take care of subscriptions and payment methods, freeing up testers from the hassle of distributing licenses among themselves.
Once the test project is ready and accessible, test engineers can begin the authoring process with Katalon Studio, which provides a feature-rich environment for test automation. With the VS Code extension already being popular among the Playwright community, users of Katalon Studio can enjoy the same benefits and features. Katalon Studio’s integrated features, such as its Object Repository, Record and Playback, and built-in keywords, help users create tests quickly and easily.
In addition to the authoring process, Katalon offers solutions for managing test artifacts, including test cases, test suites, and page objects. These artifacts can also be linked to other management tools like Jira and Xray, providing a streamlined approach to test management. As a framework, Playwright only offers simplistic tagging as a management feature, while Katalon’s custom fields and tags allow for extensive metadata association and enhanced organization of test artifacts.
The platform also provides an extensive set of test environments through Katalon TestCloud. This allows users to configure operating systems and devices for cross-platform testing and execute Playwright tests in various CI environments using Katalon Runtime Engine and Docker.
After tests are executed, the platform’s analytics module provides advanced test reports, which are automatically uploaded to TestOps and analyzed for insights, such as productivity, requirement coverage, platform coverage, and release readiness. This feature helps users to make data-driven decisions based on the results of their test runs.
Currently, the Katalon Platform has already embraced Selenium and Appium for Web and Mobile testing. With the addition of Playwright support, Katalon can become even more versatile and capable of handling a wider range of test scenarios, providing users with a comprehensive and robust testing ecosystem.
For more articles like this, visit our Product Insight corner