Automation Magento Testing with PHPUnit, Codeception + Efficient Reporting Guide

A lot of businesses around the world choose Magento shops in terms of its flexibility and scalability. They can customize the appearance, functionality, and content of their websites while keeping the shopping experience enjoyable.

Rich Magento functionality allows business owners to conduct marketing, search engine optimization or SEO, and administration functions conveniently. However, they face some challenges when added extensions start to conflict, APIs – to crash, etc. Let’s discover our Magento testing guide below to find more information about how to test a Magento-based software product to overcome the challenges 👇

What is the Magento Platform & How It Works?

We all know that Magento is an e-commerce open source software for all sizes of businesses, which are eager to sell online.  It is very flexible and customizable in terms of built-in e-commerce functionalities and third-party integrations. 

Magento’s structure is built on a Model-View-Controller (MVC) architecture so that development teams can customize nearly every aspect of the platform. Thanks to its modular codebase, they can integrate third-party modules/themes to extend the functionality of the platform and make it a custom product. In addition to that, QA teams can isolate tests during QA to identify and fix bugs more quickly and easily.

What is Magento Testing?

Magento QA testing means checking the various aspects of the website built on the Magento platform. It includes various tests to verify that the product functions correctly, meets business requirements, and satisfies user expectations. Businesses can choose the Magento platform and achieve the following goals:

• Launch a new store
• Transition from an older Magento version
• Integrate third-party systems
• Add new extensions

Types of Testing in Magento

The well-known Test Automation Pyramid illustrates us the role, value, and optimal prioritizing of different types of tests in building a test strategy.

👀 Find the common types of automation testing used in Magento e-commerce:

Functional Testing

With this type of testing, teams write tests that will verify whether a product feature provides the same output the user wants to get or meets end-user requirements. Ideally, tests are designed with a focus on the end user’s perspective.

Performance testing

This type of tests is used when you need to find out how well your product performs under pressure, to discover whether it can handle high-traffic loads without crashing or not. Performance testing includes:

• Load tests simulate real-life conditions and measure the response of the product. For instance, multiple users accessing a site at the same time.
• Stress tests determine the limits of system capacity (Simulation of Black Friday Sales, for instance 😃).

Based on performance tests, you can get essential information on how the developed software meets speed, responsiveness, and stability requirements under various conditions and helps deliver a more positive UX.

Unit Testing

Done before integration testing, this type tests individual components. Developers and QA engineers isolate these units and test them apart to verify that each part works correctly before integrating them into the larger system.

Integration Testing

This type is used to verify the interactions and data exchange between different components or modules of the product. Also, you can test how the e-commerce application uses third-party tools for customized modules. Thus, it helps QA teams identify any bugs that might arise when different components are combined and interact with each other. For example,  Payment, Gift Card, Shipping, Search, and SEO.

End-to-end testing

End-to-end tests are on top of the testing pyramid. They are typically the slowest and expensive (compared to other types of tests) but also the most representative. Since E2E tests reflect how users interact with an application, they are the most representative.

Acceptance testing

Acceptance testing is related to E2E testing; in our case, acceptance Magento testing means checking if the application meets all client requirements.

We asked the Magento community. Notice their answers.

→ Integration tests deliver the strongest confidence.
→ Build your test automation strategy considering — What is the purpose of testing, and what should it achieve?

So, if you are going to use Magento for your business, you need to consider automated testing tools and scripts to test your ecommerce website or application. It will help you launch reliable and well-performing online platforms.

What Tools to Use for Magento QA Testing?

• Magento Functional Testing Framework (MFTF). When it comes to automated end-to-end functional testing of Magento-based and Adobe Commerce projects, you can consider the Magento Automated Functional Testing Framework. It speeds up the QA process and aims to balance high code quality and reliable operations.
• Selenium. This framework is applied when you need to automate browser interactions and cross-browser testing.
• Apache JMeter. Written in Java, it allows teams to conduct performance testing by simulating loads for different scenarios and receive results in different ways – CSV files, XML files, diagrams, or graphics.
• Codeception. This PHP testing framework can be used to write tests for the Magento-based project. It supports both unit and functional testing and includes acceptance tests that simulate user interactions.
• JavaScript-based tools, namely trendy frameworks like Playwright or Cypress, verify the sophisticated Magento JavaScript frontend with E2E scenarious.

Magento Functional Testing Framework (MFTF)

The Magento Functional Testing Framework (MFTF) is a native Magento-based application officially supported by Adobe. Built on robust technologies like Selenium, Allure, and Codeception, MFTF offers stability, scalability, and customization. It makes the testing of intricate scenarios commonly found in e-commerce platforms, focusing on user and admin interactions. As e-commerce websites expand, automated testing becomes critical to ensure ongoing reliability.

MFTF supports assertions, data sets, and reusable action groups, simplifying the automation of manual test cases (conversion guides are also accessible online). Test results are presented through comprehensive Allure reports. How it works in detail: watch the Quick Guide Video

Summury

Use MFTF for standardized, Magento-native testing with strong platform support. Choose Codeception if you need more control and flexibility to cover more intricate scenarious.

Magento Testing with Codeception: 5 Steps To Follow

Prerequisites

At the very start, it is essential to prepare a testing environment for your Magento-based project. You need the following tools:

• PHP 7.4+ or 8.x
• PHP Composer installed
• Web server (Apache/Nginx) & MySQL running Magento

For Selenium Server installation:

• java -jar selenium-server-standalone.jar
• Download WebDriver (e.g. with Selenium or ChromeDriver)

Note that the PHP version should match the version of your Magento project. Run your Magento App.

#1: Set up Codeception and WebDriver

Codeception should be installed in the Magento root directory using composer through the following command:

composer require codeception/codeception --dev

In order to run your tests, you need to set up a WebDriver that depends on the browser you are using – Chrome, Firefox, Safari, etc. It can be done by downloading the appropriate WebDriver executable for your operating system and extracting the downloaded zip file, where you can find the executable file for WebDriver.

Optionally, you can install useful modules:

composer require codeception/module-webdriver --dev
composer require codeception/module-asserts --dev
composer require codeception/module-db --dev
composer require codeception/module-phpbrowser --dev

After that, generate Codeception’s default structure, specifying the path to your directory with the command:

php vendor/bin/codecept bootstrap

and

vendor/bin/codecept generate:suite acceptance

These two commands create files:

• /tests
• /tests/acceptance/functional/unit
• codeception.yml

On a file tree, the files look like:

/tests
├── acceptance
│   ├── HomePageCest.php
│   └── _bootstrap.php
├── functional
│   └── ExampleFunctionalCest.php
├── unit
│   └── ExampleUnitCest.php
├── output
│   └── (test result logs, screenshots)
├── data
│   ├── dump.sql
│   └── test_data.json
├── support
│   ├── Helper
│   │   ├── Acceptance.php
│   │   ├── Functional.php
│   │   └── Unit.php
│   └── Actor classes
├── acceptance.suite.yml
├── functional.suite.yml
├── unit.suite.yml
└── codeception.yml

#2: Configure Codeception 

Codeception should be configured through codeception.yml by setting up global paths, settings, and modules. This configuration process allows you to define where to store test results and logs, where to locate test suites and tests, and what PHP memory limit for tests should be.

#3: Create End-to-End Tests

Before creating test cases, you need to define user flows with negative and positive scenarios to be tested – adding products to the cart, checkout process, product reviews, tracking, etc. Once you have defined the user flows, you can start writing specific test cases.

So, generate a file with the HomePage name:

vendor/bin/codecept generate:test acceptance HomePage

…its path…  /tests/acceptance/HomePageCept.php

And here is a simple example below:

<?php

$I = new AcceptanceTester($scenario);
$I->amOnPage('/');
$I->see('Welcome to Magento');

More complicated User Login scenario example:

#4: Run Tests

Now you can execute the tests. To do this, you can use codecept run. Also, you have the option to choose tests for running – all, individual or specific ones. 

php codecept run -c ∼/tests/acceptance/HomePageCept.php

#5: Generate Reports & Analyze Test Results

Finally, you can view test results. They are located in the _output directory – it is the root of the project. Reports can be presented in various formats – HTML reports, XML reports, and CSV reports. In addition to that, you can view screenshots of failed tests to have the ability to analyze the execution flow.

f

Advantages of Magento Testing

Businesses of all sizes can opt for a flexible Magento platform that offers extensive customization options, a robust feature set, and a scalable architecture that can grow with your business. Let’s discover why Magento may be a good fit for you:

• You can make sure that you deliver a unique online user experience after a thorough QA process.
• You can build trust and increase loyalty among the users because you guarantee that adding new features or making changes do not introduce new bugs thanks to carrying out a well-defined automated QA process.
• When your site grows, you can easily check new features and functionality with automated tests and save a lot of time.
• When you choose the Magento platform for your business, you should know that its modular architecture allows you to separate business logic from presentation logic to streamline customization options as well as improve test automation.

Disadvantages of Magento Testing

• You need to have specific technical knowledge and infrastructure to install and configure automated testing frameworks.
• When keeping test scripts updated in terms of changes, it may introduce maintenance complexity and require additional time.
• Due to test script issues, there are false results leading to unnecessary debugging effort. 
• Simulating real-world scenarios can be complex to set up and may require specialized tools and expertise.
• You may face difficulties when you need to integrate third-party extensions with your custom code.

Useful Tips and Best Practices for Test Automation of Magento-based Products

• Before starting to test, you should have clear goals for this process as well as create a test plan. Also, don’t forget to utilize version control systems to simplify the test case management process.
• You need to keep in mind that documentation of test data – test cases and their results, especially when tests need to be repeated multiple times, will help you track relationships between requirements, test cases, and test results.
• It’s important to keep your Magento-based product and testing tools up-to-date with the latest security patches and bug fixes to protect it from threats.

Bottom Line: Ready To Test Magento App with Codeception?

With Magento’s ability to adapt to the specific needs of each business, Magento is a preferred and popular choice for those looking for long-term growth and adaptability. The platform’s user-friendly interface, great security features, and large community are very useful options for businesses. However, setting up a Magento website isn’t enough – it is imperative to perform testing to keep your website relevant and bug-free. When you push features to production, you need to catch bugs early on and avoid costly mistakes. Magento testing with Codeception will help you make the QA process more effective and hassle-free to meet users’ needs and deliver the experience they expect. Drop us a line if you want to learn more about Magento testing with Codeception.