Cucumber Testing Tutorial
on How to Organize
an E2E or Unit Framework

Cucumber is popular with modern Agile teams that practice behavior-driven development. This is due to the use of human-readable syntax for writing test cases, support for many programming languages, and an extensive community. We at decided that integrating this framework with our test management system could be useful for our customers.

Test automation allows modern Agile QA teams to reduce the time required for the testing cycle and, due to this, to perform quality software releases in the shortest possible time. Let’s see how allows you to work with one of the most popular frameworks, Cucumber.

What is Cucumber?

Cucumber is a framework written in Ruby that supports behavior-driven development (BDD), a behavior-based software development methodology. It uses the easily readable English language – Gherkin, to write user requirements. The Cucumber framework facilitates clear and concise documentation of software behaviors, which can be understood by all stakeholders.

The advantage of this approach is that Gherkin syntax is understandable to all team members, including specialists without technical knowledge e.g., (PO) Product Owner or (BA)Business Analyst. This meets one of the basic needs of modern Agile teams – technical and business team collaboration during the testing and development of digital solutions. Moreover, the integration of cucumber testing within Agile methodologies enhances the communication between developers and business stakeholders, making sure that software functionalities align closely with business requirements. Additionally, the Cucumber framework effectively aligns technical implementation with business objectives, ensuring that all team members understand the features and behaviors of the software under development.

Automation Testing With Cucumber Framework

The essence of the BDD approach is that test scripts and acceptance tests should be written before coding begins. This allows testing to be done more efficiently than it would have been done at the product release stage.

Behavior-driven development (BDD) scenarios in the Cucumber framework involve using the Given When Then structure to write. This sequence helps to describe the system from the point of view of real users.

👀 Let’s consider an example of such a scenario for testing the login functionality of a website:

Feature: Login

  I want to login on a website

    Given I go to '/login'
    And the field 'email' is empty
    And the field 'password' is empty

  Scenario: Error on empty fields
    When I click on 'enter'
    Then field 'email' should be with error
    And field 'password' should be with error

  Scenario: Wrong password
    When I enter incorrect ‘username’ AND ‘password’ values
    And I click on 'enter'
    Then I should see 'E-mail or password is incorrect’

  Scenario: Login successfully
    When I enter correct ‘username’ AND ‘password’ values
    And I click on 'enter'
    Then I am directed to the homepage

Your team can also try implement this approach in your Agile SDLC(Software development lyfe cycle), the benefits of Cucumber for testing and programming will be appreciated by all ones. Incorporating cucumber testing into your Agile SDLC not only streamlines the testing process but also ensures that the software development aligns with user expectations and behavior scenarios.

Why Do Many QA Teams Choose Cucumber?

We have tried to highlight the key benefits of test automation with Cucumber:

  1. The software requirements are formulated in simple English text format. As we mentioned earlier, Cucumber framework uses the Gherkin syntax to write test scripts, which is understandable to all team members, including every non-technical user.
  2. Support for multiple programming languages. Unlike many other frameworks that support a single language, the Cucumber tool supports Ruby, Java, JavaScript, Scala, Perl, and other programming languages. This allows you to write code along with Cucumber framework in your preferred language without worrying about whether or not you can use the framework. The versatility of Cucumber in supporting various languages makes it an ideal tool for behavior-driven development, as it accommodates a wide range of technical environments and team skills.
  3. Written code can be reused. All Cucumber feature files contain clearly structured Given When Then test scenarios. This makes it easier to reuse the code, speed up the software testing process, and get the finished software product to market. Furthermore, the integration of cucumber testing with other tools and frameworks in the development pipeline can further streamline and optimize the entire software testing lifecycle. The reusability of code in Cucumber is especially beneficial in the context of behavior-driven development (BDD), where similar user scenarios may recur across different features or applications.

Sync Cucumber Tests With Manual Testing

We consider the needs of modern Agile teams who choose to work with the BDD framework Cucumber, so we implemented the possibility of integration with automated tests written with Cucumber in our TMS. This integration is based on the Three Amigos approach, meet more detail: What are 3 Amigos in Agile? by following the link to the article on our Blog.

Sync Cucumber Tests With Manual Testing

Synchronization makes BDD tests available to all team members and allows them to work together on the project. See below how it works step by step.

Getting Started Cucumber From Example (ready boilerplate)

Our TMS supports integration with the most popular software testing tools, including WebdriverIO, Playwright, Protractor, TestCafe, CodeceptJS, Codeception, and Jira project management system in the next combination with Cucumber framework:

  • Cucumber
  • Cypress+BDD
  • Playwtight+BDD
  • WebdriverIO+BDD
  • CodeceptJS+BDD
  • Codeception+BDD
  • TestCafe+BDD
  • Java+BDD works through JUnit XML reporter
  • etc.

Our tool is completely flexible, as you may see. This tutorial has all the information you need to work with BDD successfully. Particularly for cucumber testing, our platform ensures seamless integration and execution, providing an efficient way to manage and run your BDD scenarios. Below we will focus on the Cucumber sample. If you use another tool, change the testing tool you need.

#1 Create A New Test Project

Create a new project in TMS in one click if you have ready test automation framework already, like in the picture. You should choose the BDD option for the BDD project logically.

Create A New Test Project

After then import the feature file. To do this, go into the project dashboard and select the Import Project from the Source Code menu item.

Import Project from the Source Code menu item

#2 Generate API Key And Import Autotests

Select the framework and project language. TCMS recognizes your laptop OS and generates a command with API key according to it. You should copy this command and execute it on CMD. This one uploads your Cucumber tests from the source code to test management.

Generate API Key And Import Autotests

End your importing by pressing the Finish button. At the same time, look at the console, how the importer is analyzing code accurately and displaying the number of founded tests in the test framework.

Import project from Source Code

If you enter your project, you will see these tests. You can view their structure by expanding the tree. You may enter your suites and check test cases inside on the bird’s eye.

First BDD project

In addition to importing an existing feature file, you can create a new manual test case. To do this, select the Create Suite menu item, and in the window that opens – Create Test. You can create tests one at a time or use the Bulk function and write all the tests you need at once.

Create Suite menu

If you copy test case from the code editor, as I did, you can auto-format it to bring it a tidy appearance.

In writing manual test cases, users of our TMS are helped by another feature – Intelligent auto-complete steps. After the feature file is synchronized, all existing steps in it are automatically added to the Steps database. This is very convenient because when writing new test cases, they can be reused by selecting them from the open list.

#3 Organize your Cucumber tests in the best way

If you need to structure test cases, you can reorder them or use Bulk Action, which allows you to reorder tests, create new test suites, move test cases to another test suite, or delete certain tests.

With Project Timeline, or Pulse, you can analyze all the changes you have made to your project. You can see what was changed, by which user, and when, and, if necessary, undo the wrong action with the Rollback or Restore to Previous Version button. Moreover, this functionality proves particularly useful in cucumber testing, where tracking the evolution of test scenarios and their outcomes is crucial for continuous improvement in BDD practices.

#4 How Do I Create A Report In Cucumber Framework?

For Cucumber framework, as for others, there is a Reporter, which must be installed in your test project. Just copy and paste this command into your command line.

How Do I Create A Report In Cucumber Framework
After that, you can run a test run.

test run

During the test run, you will have access to a real-time results report. You will see detailed information about what went wrong if any tests failed. Here you can also see the script code, existing attachments, etc.

real-time results report

If you run a manual test, you can also get a report on its results. You have the opportunity to view the status of each test case or to examine all the details: check what failed. You can also filter the results you want.

Thus, test management allows you to control the performance of manual and automated tests in one place.

The TMS provides additional functions for user convenience. One of them is rich Analytics for your Cucumber testing framework:

Analytics for Cucumber testing framework
In-depth analytics, based on detailed reports of test results, are available on the Project level as well as the Global level (implemented to satisfy QA managers’ needs). Thanks to this feature, you can track test coverage and check if the project has flacky tests, slow tests, ever-failing tests, etc.

Running Cucumber Test On CI\CD

You can set up the CI\CD pipeline and run all or individual Cucumber tests. supports seamless integration with popular CI\CD tools: GitHub, GitLab, Jenkins, Bamboo, and CircleCI. This enables continuous testing, real-time reporting, and notifications at every stage of the software testing lifecycle. Especially for cucumber testing, this integration facilitates the execution of BDD scenarios in a continuous integration environment, enhancing the efficiency and effectiveness of your testing strategies.

One key thing that you must decide is: in which way do you want to manage the Cucumber scenarios? Do you want to use TMS or Advanced Jira Plugin directly? The Jira plugin allows you to do everything you do in the test management system. All actions are synced between them. between It was done in order to involve non-technical specialists in the technical part of the software testing process.

Comparison table: Cucumber vs. Playwright vs. TestCafe

Cucumber Playwright TestCafe
Multilingual framework (supports Ruby, Java, JavaScript, Scala, Perl, and other programming languages) Multilingual framework (supports JavaScript, Python, Java, and C#) Supports only Javascript
Supports parallel testing thanks to the TestNG and Maven modules It is possible to run tests in parallel Parallel mode is available for simultaneous test runs
Actively supported Very actively supported over the past few years Actively supported
Ruby-based framework JavaScript-based automation testing tool JavaScript-based framework
Cucumber framework is not a browser automation tool, but you can use Cucumber with Selenium WebDriver, which supports Google Chrome, Internet Explorer, Safari, Opera, Firefox, and such OSs like Windows, Mac, Linux/Unix. Supports over 40 popular software platforms and OS Supports Google Chrome,
Internet Explorer,
Microsoft Edge,
Mozilla Firefox,
Google Chrome mobile,
Safari mobile, and also three OSs: Linux, Windows, and macOS.
Gherkin language is used for writing test cases – plain English text, understandable to all team members If you don’t want to create tests by hand, you can use the Codegen function, which will generate test cases in the programming language of your choice The TestCafe tests are Node.js scripts
Used for end-to-end, integration and unit testing Preferably used for end-to-end automated testing of web apps Mainly used for end-to-end testing

We hope you were able to learn a lot of useful information from this Cucumber framework tutorial and take advantage of the integration features that are implemented in our TMS. Namely, on How to organize your Cucumber tests and get a reach report and analytics in a simple way.

The content is available also in video format if you convenient reproduce the steps to visualize your tests and get an informative report by watching.

Frequently asked questions

What is Cucumber Framework? Testomat

The Cucumber framework is an open-source tool that supports Behavior Driven Development (BDD). By using Gherkin syntax, Cucumber bridges the gap between business requirements and technical implementation. It allows teams to create automated tests with plain language in Gherkin syntax, which makes it easier to describe the behavior of the application in a way that can be understood by all team members, including business analysts, developers, testers, and product owners (non-tech teammates).

How does Cucumber work? Testomat

Cucumber works by allowing you to write human-readable tests in Gherkin syntax, mapping those tests to step definitions written in code, and executing them to verify the behavior of your application.

The main Cucumber essences are:

  • Feature Files: tests in Cucumber are written in Gherkin language and saved in files with a .feature extension. Typically a feature file contains one or more scenarios.
  • Scenarios: each scenario describes a specific situation or behavior to be tested. Scenarios are written using a Given-When-Then structure to describe the preconditions, actions, and expected outcomes.
  • Steps: each step in the Gherkin scenarios needs to be connected to code that performs the described actions. These are written in a programming language such as Java, JavaScript, Ruby, or Python.
  • Annotations: Step definitions use annotations (like @Given, @When and @Then) to map Gherkin steps to corresponding methods.
  • Run the Tests: the Cucumber framework runs the feature files and executes the corresponding step definitions. It matches each step in the scenario to its respective code and performs the actions.
  • Reporting: after executing the tests, Cucumber provides detailed reports showing which scenarios passed and which failed, along with any error messages.
  • Test Automation Tools: Cucumber can integrate with Selenium, Appium, and many other automation tools for interacting with web and mobile applications.
  • Continuous Integration (CI): Cucumber framework can be integrated into CI pipelines using tools like Jenkins, GitLab CI, or CircleCI to automate test execution as part of the build process.

How BDD works in Cucumber Automation? Testomat

Cucumber is a popular tool that, enabling teams to create automated tests that are easy to understand and maintain. Here’s how Cucumber works in automation:

  1. Cucumber automation starts from defining an application behavior in plain language using Gherkin syntax. This makes the requirements clear and understandable to all stakeholders.
    → Feature: Describes the feature under test.
    → Scenario: Describes a specific situation or behavior to be tested.
    → Given-When-Then: Defines the steps of the scenario.
  2. Writing Step Definitions: it is a bridge the gap between the Gherkin steps and the underlying code. Each step in the scenario is linked to a method that performs the necessary actions or verifications.
  3. Running the Tests: tests can be executed from the command line using Maven or Gradle. Cucumber reads the feature files and executes the corresponding step definitions. The results show which scenarios passed and which failed, along with detailed error messages if any. Cucumber can generate various reports, including HTML reports, for easy review.
  4. Integrated Development Environment (IDE): many IDEs like IntelliJ IDEA, Eclipse, or Visual Studio Code have plugins to run Cucumber tests.
  5. Integrating with CI\CD pipelines: Cucumber can be integrated into CI\CD pipelines to automate the testing process. Tools like Jenkins, GitLab CI, or CircleCI can be used to run Cucumber tests as part of the build process, ensuring that any changes to the codebase do not break existing functionality.
Create first BDD project
Turn your Jira stories into BDD features as well as test cases into BDD scenarious 👇
Follow us