CircleCI integration with test management

In a world where organizations are looking to simplify and accelerate their product release with less effort and greater reliability, new challenges have emerged.

Currently, the testing process has become a significant bottleneck for development and QA teams to perform tasks effectively. Avoiding these bottlenecks is a business imperative. If your team is not already empowered to react to this problem, you may be leaving your business vulnerable. Given that, the importance of integrating automated testing directly into CI\CD pipeline has never been more apparent.

Whether you’re working on an old one or launching a new one, improving the quality of your software releases with the right CI\CD tools is a must. This will help you significantly strengthen feature quality and streamline development possess. Read our article to learn more details!

What’s CI\CD pipeline?

Continuous integration (CI) and continuous delivery (CD) are two essential practices used in Agile and DevOps that help development teams integrate their code early into a shared repository and make sure that new features reach customers as quickly as possible. Being a crucial component for modern software development and agile teams, it helps boost team productivity, reduce failures and shorten the release cycles.

What’s CircleCI?

Built in Clojure, the CircleCI platform provides continuous integration and delivery processes for development teams and helps them efficiently release code as well as automate development, testing and deployment hosted on a GitHub repository. It offers the following types of setups:

• Hosted service: users can log on to that hosted platform and run tests, all their builds, and pipelines there.
• On-premise installation: users are able to install and maintain it on their own infrastructure.

What are key features of CircleCI?

Here we are going to overview some key features that make CircleCI tool great for the broad software development community and testing teams.

• ‍Insights Dashboard: this feature allows monitoring all continuous integration (CI) and continuous delivery pipelines and finding all the essential information about workflow status, duration, etc. Additionally, teams can track the status of the jobs and pipeline, identify opportunities to cache, insert parallelize and optimize consumption in the testing process.
• Orbs: this feature provides reusable, simplified commands and helps development teams work with commonly-used services such as Google Cloud, AWS and Azure.
• Parallelism: this feature helps teams reduce the time needed for tests to finish – you can change the number and types of containers as well as determine the number of resources that will be used to finish the job.
• Jobs: this feature allows teams to run jobs simultaneously – you can execute tests async without the need to run them independently.
• Workflows: this feature allows development teams to define, design and keep under control job executions.
• Automated Deployment: this feature provides a quick time-to-market strategy by deploying pipelines to multi cloud environments.

What are the benefits of CircleCI integration?

Integrating the CircleCI tool into the software development process provides multiple benefits to the whole process. Let’s discover the benefits you can reap from CircleCI integration:

• CircleCI is powered with a clean and simple UI that makes it quicker and more responsive for users.
• CircleCI integration is highly secure – all variables are encrypted.
• CircleCI offers a great number of infrastructure options across operating systems and provides a quick move with an easy setup.
• CircleCI allows teams to integrate it easily with software repositories such as Bitbucket/itHub/GitHub, etc. with ease.
• CircleCI provides automated parallelism to speed up the deploying code of multiple executions.
• CircleCI runs on an online server and avoids the need to have internal dedicated server support to start working.

How does CircleCI work?

A software repository on a supported version control system needs to be authorized and added as a project on circleci.com. CircleCI integrates with GitHub, GitHub Enterprise, and Bitbucket. Every time a code commit is made, CircleCI creates a pipeline. Every code change then triggers automated tests in a clean container or virtual machine. CircleCI runs each job in a separate container or virtual machine.

CircleCI then sends an email notification of success or failure after the tests are complete. CircleCI also includes integrated slack and IRC notifications. Following the completion of tests, CircleCI provides detailed CircleCI test results that help in assessing the success or failure of the tests.

Why development teams opt for CircleCI?

Choosing the right CI\CD tool is crucial for your DevOps pipeline. It not only speeds up your product delivery but also makes continuous integration smoother and easier. According to a recent official CircleCI State of the delivery report, 50% of development teams that select CircleCI tool for release build and deliver elite-level products. Being a completely automated CI\CD tool, CircleCI provides easy setup and maintenance without any difficulties.

If you need a quick move in your software project, this CI\CD tool is a go-to choice and a great example of quality:

• CircleCI serves around 30000 clients and is capable of running a million tasks per day.
• CircleCI offers performance-based scaling options.
• Incorporates SSH into the build and test runs to debug.
• CircleCI runs every task as a new container which prevents stale build data from causing issues.
• CircleCI announces the end of task execution via Email Notification.
• Another important thing is that you don’t need additional infrastructure to maintain your CI (continuous integration) environment.
• Moreover, the parallel execution of builds makes CircleCI software a good option for your development and testing teams. If integrated with the test management system, your development teams can focus more on code accuracy and better coordination as well as keep a close eye on the overall performance of the testing process.

How to integrate CircleCI with test management

In testomat.io testing tool you can trigger jobs on external CI systems using REST API. If only a specific test or suite is executed, tests IDs or suites move to the job. With test runner, all tests are stored by their IDs and executed as a subset of tests. During the ongoing process, a report is sent to test management through the reporter option in real-time even if just one test was finished. Here we show step-by-step the CircleCI integration with our test management system.

#1: Start a Project and create Test cases

When connecting the CI server to the test management system, firstly you should create a project in and find automated tests in a repository and import your automated tests into the project.

#2: Assign Testomatio IDs to automated tests

Here you can discover how to assign testomat.io Ids for imported tests according to the frameworks used: use the check-tests command for importing tests with --update-ids for JavaScript frameworks.

TESTOMATIO={apiKey} npx check-tests <framework> <pattern> --update-ids

use check-cucumber command with --update-ids command for Cucumber framework.

TESTOMATIO={apiKey} npx check-cucumber <pattern> --update-ids 

With those commands, you can update your source code. Take into account that all ID changes mustn’t include mistakes. You need to check them before pushing a commit to the repository.

#3: Configure Continuous integration settings

Connect the test project to your CI\CD pipeline by choosing the Continuous integration tab on Project Dashboard.

After then, choose CircleCI and fill in the needed settings. After then Save them.

Pay attention to the following checkbox configuration in the Input variables tab have to be checked:

• Send Run ID as ‘run’ input (required for scheduled jobs)
• Send Testomatio API key as ‘testomatio’ input
• Send Testomatio Server URL as ‘testomatio_url’ input

Check your connection by clicking on the Test connection button. The successful message should appear.

#4: Create a new job in CI to build and test

CircleCI uses path/file .circleci/config.yml for configuration. That’s why you need to create a workflow used solely by testomat.io The workflow should start on workflow_dispatch event with the following input parameters of your code:

parameters:
  testomatio: 
    type: string
    default: ""

  run: 
    type: string
    default: ""

  testomatio_url: 
    type: string
    default: ""

  grep:
    type: string
    default: ""

The job should include a step where the test runner is executed with –grep option for the Testomatio environment variables to be passed in.

      - run:
          name: Run tests
          command: 
            cd $CIRCLE_WORKING_DIRECTORY
            npx codeceptjs run --grep "<< pipeline.parameters.grep >>"
          working_directory: ~/codeceptJS
          environment:
            TESTOMATIO: << pipeline.parameters.testomatio >>
            TESTOMATIO_RUN: << pipeline.parameters.run >>
            TESTOMATIO_URL: << pipeline.parameters.testomatio_url >>
          

#5: Execute tests and view results

Once you successfully set up our Circle CI pipeline, you can execute your automated tests by opening a test, selecting Run in CI and clicking the Launch button.

Moreover, you can now report your test results to the test management tool. This will help you track and view all test results with ease, and make them accessible to the entire team – stakeholders, BAs, PMs, development and testing teams.

Additionally, you can define what tests are slow, failing, or flaky and make changes. This will help them make data-driven decisions and appropriate improvements. To sum up, Circle CI integration is a good example of taking a positive effect on the quality and performance of the product being tested at any stage.

Bottom line: Want to let your CI\CD flow with CircleCI integrated?

Implementing a CI\CD pipeline has become a requirement in our agile-driven world. If properly incorporated, development and testing teams can discover bottlenecks in their early stages and identify issues more quickly.

Additionally, it helps execute tests faster, avoid similar test failures, detect potentially affected tests and get a crystal picture of test results. With the right tools in place — which is facilitated both by a well-aligned development team and a well-defined testing strategy, companies can take significant advantage of the process. They not only can identify and resolve bottlenecks today but also during future periods of business growth.

Having that in mind, you will need the CircleCI integration with testomat.io test management system to reflect the new messaging of the IT world.

Configure the next integration and run your automated tests on CI\CD:

• Playwright Testing Tutorial on How to Organize an Advanced (Scaled) (e2e/unit) Testing Framework
• The Ultimate Cypress Tutorial: How to Organize your Advanced Testing Framework
• Codeceptjs testing tutorial on how to organize an advanced e2e testing framework
• Cucumber Testing Tutorial on How to Organize an E2E or Unit Framework

If you have any questions, contact our CI\CD experts who will help you in building CI\CD process that meets your DevOps needs.