Introduction to Serenity BDD Framework – Tutorial

When developing and testing software products, misunderstandings and unmet expectations could happen in the teams and lead to duplicated work, wasted time, and lowered productivity throughout the software development lifecycle. When the non-technical business stakeholders don’t know how to build an app, and technical teams don’t know what exactly the wants and needs of the end users are, there is a disconnect between business and development.

Here’s where the Serenity BDD framework could close the gap. When utilized properly, technical teams and non-technical stakeholders can be on the same page about what is going on in the software project.

In our Serenity BDD tutorial, you can get answers to questions like:

— What is Serenity BDD?

— Why should teams use Serenity BDD?

— How to solve challenges with best practices and tips?

… as well as find more useful information on BDD tools 🛠️

Serenity Framework Overview

Serenity is an open-source testing framework designed to automate the testing of Java-based applications. With this systematic method, testing, QA, and development teams can create automated tests and check the quality and dependability of the produced code.

Furthermore, combining the principles of Behavior-Driven Development and the Screenplay pattern helps technical specialists write readable and scalable tests.

👀 Let’s reveal some key features of the Serenity framework:

• Feature Files. Written in Gherkin syntax, these files provide information in a human-readable way so that everyone can understand how the app should work.
• Scenarios. Scenarios embedded into the files represent specific behavior examples, structured into Given-When-Then steps.
• Step Definitions. Step definitions provide the connection between feature files in the Given-When-Then syntax.
• Actors. They are users or personas that interact with the system and perform by applying their knowledge and skills needed for various actions.
• Tasks & Actions. Tasks define the actions that should be performed by users – scrolling through a page, clicking buttons, or submitting a form. With the Screenplay pattern, tasks help create more understandable test code.
• Questions & Assertions. They help you compare the actual app state to your expected results. When using questions, you can validate the application behavior during test execution quickly.
• Test Execution. With a test runner included, Serenity allows teams to execute defined scenarios and produce detailed reports. Throughout test execution, it logs steps, takes screenshots, and tracks performance metrics.
• Reporting. Based on test reports, teams can analyze outcomes, identify failures, and monitor testing progress in order to make changes and produce better testing results.
• Integrations with Selenium WebDriver and Cucumber Libraries. When integrating with mentioned libraries, you get the necessary browser control and interaction features for testing web applications.
• Test Data Management. With Serenity, you get tools for managing test data, creating data sets, and handling dependencies as well as using different data sets each run.

Serenity BDD: Definition, Key Features & Architecture

Previously known as Thucydides, Serenity BDD is used as an open-source testing framework and is designed to simplify the way of writing automated acceptance tests and regression tests. With its help, developers and testers can create complex tests and apply well-illustrated test reporting and living Serenity BDD documentation options.

With the Gherkin language, an ordinary language parser, when complex testing information might be scripted almost in English. Also, it is important to mention that Serenity BDD not only reports on what requirements were tested, but how they were tested. It truly documents what your application does with optional screenshots that present step-by-step details of how a test played out.

Key features of Serenity BDD are presented below:

• BDD principle. With a specific Given-When-Then format, it helps create test scenarios written in human-readable statements for better understanding.
• Test Automation. Thanks to this feature, you can carry out automation for web, mobile, and API tests.
• Report Generation. With Serenity BDD, you can visualize rich and informative test reports based on test results and identify areas that need to be improved.
• Parallel Test Execution. When running multiple tests in parallel, it generates unique reports for each test and provides a clear overview.
• Test Management Integration. This feature allows you to integrate it with popular test management tools (e.g. testomat.io).
• Custom Reporting. With this feature, you can choose how to format and aggregate test results by customizing reports.

What Are Key Components of Serenity BDD Architecture?

• Page Objects. They include all available actions within the entire application that are performed with certain page elements.
• Steps. A place where web elements and methods are declared and can be used for further manipulations.
• Requirements. They specify what the app must do and include acceptance criteria to determine whether the app meets the required functionality.
• Tests. Designed to exercise the app functionality, they are specific scenarios that check if the app meets the acceptance criteria.
• Reports. All the results of the test runs can be found here and analyzed.

Why QA teams need BDD framework

BDD framework can bridge the gap between software engineers, QAs, and business stakeholders. With BDD, QA teams can:

→ Talk with technical teams and business stakeholders about user needs and software requirements in a business readable language (Gherkin) created especially for describing users’ behaviors.

→ Create user stories and scenarios to define acceptance criteria for specific software features that are understandable to all stakeholders.

→ Test business scenarios from the start and identify bugs early in the development cycle.

→ Turn business requirements into a clear format for the tests that should be executed to minimize or avoid bugs and errors.

Why use Serenity BDD in automated testing

The BDD approach fits well with test automation because the scenarios created in business-written English can be put in a feature file and used for test automation. The technical people on the team can implement each scenario in the tech specification files and take full advantage of it:

• With its clear reporting, teams can collaborate and understand the overall testing progress more easily.
• When applying plain Gherkin language, teams can describe test scenarios so that both technical and non-technical teams can understand the expected/actual outcome of automated acceptance testing.
• Serenity BDD as a powerful automation framework helps streamline the test automation process, saving time and effort for the teams.
• It generates detailed reports from the box that provide insights into automated test execution, failures, and performance metrics.
• With the App’s growth, Serenity’s modular approach makes it easier to maintain and update automated test cases.
• Its ability to create complex test scenarios guarantees comprehensive automation test coverage.

Linking User Stories With Tests Through Serenity

In Agile projects, teams mainly register requirements through User Stories. They represent a description of a business need in requirement through a user perspective while written in BDD acceptance criteria to check if those business needs or requirements are met. The basic format of a User Story is simple:

Who: As a [user type]
What: I want [action to perform]
Why: So that [the desired outcome]

In Serenity BDD, you can link User Stories with test cases. It can be done through annotations or tags in order to associate specific test cases with particular user stories.

Also, you can use test case management systems like testomat.io In the mentioned TCMS, you can link your requirements to test cases within the system and manage them there.

Another variant is to customize Serenity’s reports to display information about linked user stories or configure the Serenity reports by including links to your test management or other relevant documentation.

How to Get Started with Serenity BDD Framework – Tutorial

Requirements and Steps

When working with Serenity BDD, you need to take into account that requirements are organized into three levels – capabilities, features, and stories. This means that each capability consists of many features, and features are explained in detail by user stories.

Also, Serenity BDD framework applies the following concepts:

• Story represents a small workflow unit business functionality.
• Scenarios display how the system functions.
• Steps are used to represent actual behavior using classic Given-When-Then keywords.

In a Gherkin file, an app feature needs to be described for testing. The file should contain the Feature keyword at the very beginning with the feature name going after it on the same line. An optional description may cover a few lines below.

Here is a simple Gherkin document example:

Feature: User Authentication

Scenario: Logging in with the correct credentials
  Given the user has an existing account
    And the user is on the login page
   When the user enters a valid username and password 
   Then the user is redirected to their dashboard

Scenario: Logging in with the incorrect password 
  Given the user has an existing account
    And the user is on the login page
   When the user enters a valid username and an incorrect password 
   Then the user sees an error message

Setting up Serenity BDD

Before getting started with Serenity BDD, you need to install:

• JDK 1.8 or higher should be fine (you can choose Linux, Mac or Windows)
• Java Development Environment such as IntelliJ or Eclipse
• Maven/Gradle build tools, we used Homebrew and Maven now for this purpose
• Also, you need a basic understanding of Git.

First Test with Serenity BDD

Now we will be using the Serenity BDD with JUnit template project to get a simple project up and running quickly as recommended on the GitHub repository page.

1. 1. Download the Serenity JUnit Screenplay Starter project, unzip it
   2. Enter into it cd and execute framework with the next command:

      mvn clean verify

   3. After the tests have been completed, Serenity BDD will generate a test report in the target/site/serenity directory. If you open the index.html file in your browser, you can view the test results.

In the build-in Serenity test report, you can find out this:

• The passed/failed/ignored/skipped tests
• Duration of tests.
• Images and videos of test runs.
• Comprehensive details about the test data.

Testing with Serenity BDD

As you can see Serenity BDD allows writing tests in a Given-When-Then format, translating user actions into the GWT format can you reflect the Steps. As tests go along with @Steps, you need to create a test that tells a simple user story carried out using certain Steps.

src
├── test
│   ├── java
│   │   │   ├── pages
│   │   │   │   └── GoogleSearchPage.java
│   │   │   ├── steps
│   │   │   │   └── GoogleSearchSteps.java
│   │   │   ├── steps
│   │   │   │   └── GoogleSearchStepDefinitions.java
│   ├── resources
│   │   ├── features
│   │   │   └── GoogleSearch.feature

Let’s create one more test, we will search for a request by entering a keyword on the Google page:

File GoogleSearch.feature

Feature: Google Search
  As a user, I want to search on Google
  so that I can find relevant information.

  Scenario: Search for a keyword on Google
    Given I am on the Google search page
    When I search for "Serenity BDD"
    Then I should see results related to "Serenity BDD"

Step Definition File GoogleSearchStepDefinitions.java

package com.example.steps;

import io.cucumber.java.en.Given;
import io.cucumber.java.en.When;
import io.cucumber.java.en.Then;
import net.thucydides.core.annotations.Steps;
import com.example.steps.serenity.GoogleSearchSteps;

import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;

public class GoogleSearchStepDefinitions {

    @Steps
    GoogleSearchSteps googleSearchSteps;

    @Given("I am on the Google search page")
    public void iAmOnTheGoogleSearchPage() {
        googleSearchSteps.openGoogleHomePage();
    }

    @When("I search for {string}")
    public void iSearchFor(String keyword) {
        googleSearchSteps.searchForKeyword(keyword);
    }

    @Then("I should see results related to {string}")
    public void iShouldSeeResultsRelatedTo(String keyword) {
        assertThat(googleSearchSteps.getFirstResultText(), containsString(keyword));
    }
}

Steps Class GoogleSearchSteps.java

package com.example.steps.serenity;

import com.example.pages.GoogleSearchPage;
import net.thucydides.core.annotations.Step;

public class GoogleSearchSteps {

    GoogleSearchPage googleSearchPage;

    @Step
    public void openGoogleHomePage() {
        googleSearchPage.open();
    }

    @Step
    public void searchForKeyword(String keyword) {
        googleSearchPage.enterSearchKeyword(keyword);
        googleSearchPage.submitSearch();
    }

    @Step
    public String getFirstResultText() {
        return googleSearchPage.getFirstResultText();
    }
}

Page Object Class GoogleSearchPage.java

package com.example.pages;

import net.serenitybdd.core.pages.PageObject;
import net.serenitybdd.core.annotations.DefaultUrl;
import org.openqa.selenium.By;

@DefaultUrl("https://www.google.com")
public class GoogleSearchPage extends PageObject {

    private By searchField = By.name("q");
    private By firstResult = By.cssSelector("h3");

    public void enterSearchKeyword(String keyword) {
        $(searchField).type(keyword);
    }

    public void submitSearch() {
        $(searchField).submit();
    }

    public String getFirstResultText() {
        return $(firstResult).getText();
    }
}

To run your Serenity BDD tests, you need to execute the following command in the root directory of your project again

mvn clean verify

Customize Serenity Test Results Report With Testomat.Io

When tests have been executed, you can download your Serenity results report in .XML format, also generate a report and synchronize your automation testing with manual testing. It provides technical and non-technical teams with more detailed test analytics based on retrieved test results. With its help, you can select widgets’ relevance and organize them in the preferred order to conduct data analytics effectively. With the intent of meeting user requirements, you can select analytics widgets for displaying – test coverage, automation rates, defect distribution, or execution timelines. It will let teams apply the option to derive required information and reach their testing goals more easily.

When using testomat.io TCMS, you can group test results into Assignees, Labels, Priorities, Custom Statuses to grasp the main points of testing efforts. Also, you can display test outcomes organized by test suites to understand their quality.

To do this, first create a new project in the test management system and make an import as it looks like in the next image:

Challenges, Tips, and Tricks in Serenity BDD

When working with any framework, teams may recognize the limitations and face some challenges:

• It is hard to work on the project if there is a misunderstanding between the person who produces the automation code and the person who writes the feature files.
• There is a need to install and update libraries and add-ons for mobile testing as it only supports web and API testing.
• Your teams require strong technical knowledge to integrate additional tools into the development workflow and adapt in accordance with their needs.
• It is inefficient for a short project as teams need a lot of time and effort to create scenarios and maintain the files.

To solve these problems, you need to follow tips and tricks and then get the most out of Serenity BDD:

• You need to remember that each feature should focus on a single user story or functionality.
• You need to create names for tests that describe what the test case does.
• You need to write clear Gherkin scenarios and keep your steps organized.
• You can use the @Step annotation to mark individual test steps and the @WithTag– to categorize tests.
• You need to regularly review and update test reports to make sure they are accurate.

Bottom Line

With Serenity BDD framework, programmers, testers, and business stakeholders are on the same page on the features that should be developed, without bridging the gap between techies and non-techies.

When people have clarity about what feature is going to be developed and have enough understanding of why it should be done, there is a possibility to deliver a software product that meets or exceeds users’ expectations.

👉 Drop us a line, if you need more information about Serenity BDD and testomat.io integration.