The main goal of software testing is the delivery of a bug-free product. Automation testing helps a lot in achieving this goal. Moreover, automated testing reduces regression testing time and allows adopt CI\CD pipeline to deliver software rapidly. But, how do you understand that product and its features are on track for release?
Test reporting and analysis can give the feedback that your app is of an acceptable level of quality.
Every time an automated test script runs during the test life cycle, the Automation QA team need to generate a test execution report of these tests. Test Management System testomat.io provides test result reporting with rich details. These test reports represent data in a clear form as to the business side (product manager, analysts) as well as developers.
Сheck test results of the following types of testing:
- End2End tests
- API tests
- Integration tests
- Unit tests
And successfully organise:
- Functional testing
- Smoke testing
- Regression testing
- Exploratory testing
Test reporter vs test runner and test report
That’s the description of test report implementation in our test management solution. So let`s consider the next entities in which way they work together!
- Test importer – at first, you need to import automated tests into a project. Test importer connects and makes the import of tests to your project in test management system.
- Reporter – is based on the open-source library. Reporter is used to output test run reports in a certain format. Before running tests, you should select a reporter from the dropdown list of testing frameworks and the type of your project, mean classical project or BDD project.
- Test runner – is something that allows us to run the tests. Test runner calls automated tests or groups of tests from the source code and then executes them and writes the test results to the console.
- And the Test report is a result of the output of your tests. Rich HTML report.
Test importer, test reporter and runner start from the command line (Command Line Based Tool) with a few simple commands. We implemented a convenient UI/UX interface with CLI command tips on each step. Just copy and paste it into your console without any special settings and get a response. Also, you are able to execute tests and receive real-time test reports with Continuous Integration services, such as GitLab, GitHub actions, Jenkins, Bamboo, CircleCI and many more. And seamlessly integrate Test importer, Test reporter through API in your custom dashboard.
Testing tool support program languages:
- JavaScript
- TypeScript
- Java
- PHP
- Ruby
Test reporter works for the following test frameworks:
- CodeceptJS
- Codeception
- Cypress.io
- Playwright
- Webdriver.io
- Jest
- Mocha
- Chai
- Protractor
- Jasmine
As you know, these reporters can be specified via configuration on the command line. Learn how to install and configure reporter for your project within our test management solution, see on Docs.
*Alternatively, through JUnit integration XML file you are able to generate reports for any language or other test framework. Meet all our integration by the following link
How to create a test report?
Reporting capabilities is available after registration in the test management system.
Components & structure of test report
The overview report page is the HTML page that depicts results within several default widgets. Widgets represent the state of the test result and test environment.
- Pie charts
- Graphs
- Suites and test cases tab
- Heatmaps
- Progress bar
- Environment
- Test run history logs
- Analytics
Let’s mention a few related first-class test reporting features:
- Real-time report – test reporter sends information immediately. Possible access to test report results once even 1 test is executed!
- Public report – instant sharing of test results to stakeholders. Public report won’t contain security-sensitive information.
- Analytics Dashboard – gives a possibility to track such testing metrics as Flaky tests, Slow Tests, Automation Coverage, Defect Coverage, Tag statistic etc.
- Aggregated analytics across different projects – analytics dashboard collects data of all your testing projects within a company.
- Flaky test management – easily detected flaky tests! And with a flexible “flakiness” configuration decide which tests should be flaky yourself.
- Stacktraces and exceptions – exceptions inform us about the presence of error. Stack trace, meaning Stack of Call directly points on the part of code that caused this error.
- Unlimited artifacts storage S3 private cloud – test artifacts are uploaded to S3 Object Storage bucket on AWS, DigitalOcean, Azure, or Google Cloud. So, by using external storage you get full control over artifacts.
- Jira integration – advanced Jira plugin provides end-to-end functionality to link tests to requirements user stories. Non-technical teammates and QA engineers can view, edit, run scenarios also reproduce bugs directly from Jira. Meet all test management features
Key benefits testomatio test reporter
- Test reporter is really fast. You can easily import and execute thousands of tests.
- Clean and easy-to-understand UX/UI with autogenerated tips.
- Provides synchronization with manual tests.
- Always relevant reports of test results and progress throughout the testing process.
- Comprehensive advanced analytics in the Test report and Analytics Dashboard.
- Allows organizing CI\CD pipelines for Agile teams.
