Work with thousands of test cases – torment or pleasure 🤔

Proper documentation and maintenance of test cases are essential to ensure software testing is thorough, accurate, and effective in detecting defects and ensuring high-quality software products. Moreover it is important for projects that are growing up intensively. Where new QA professionals hire.

Test case

This document is one of the primary documentation for QA Engineers after bug reports. This artifact will help track which functionality has already been tested. Also, all ideas are recorded in a test case. This allows the engineer to free up his capabilities for exploratory testing. As a result, you can find additional defects and better understand the system.

Also, this document is beneficial for newcomers to the project. Thanks to this, the new QA can understand the ideas and purpose of the project and what needs to be paid attention to during testing.

However, the larger the project, the more test cases there will be. Over time, they become difficult to maintain. And some test cases begin to follow the principle of the paradox of pesticides, which makes their further use impossible.

👉 Let’s clarify how to avoid the paradox of pesticides and maintain a lot of test cases!

Selecting tеst management system

Before writing test cases, need to understand where should it be located.
First of all, it is necessary to analyze the project itself:

→ What tools will be used for it?
→ How many people are on the team?
→ How complex the project is?
→ Whether it is possible to use checklists in some functionalities?
→ Whether there will be automation?

After clarifying each point, time to choose the test management system TMS. For example, there are the following:

• Zephyr – add-on of JIRA
• TestRail
• testomat.io

Each test management tool has its advantages and disadvantages. In my opinion, the pros and cons of each tool are as follows:

Zephyr:

• Good that it’s directly in JIRA
• Free edition for ten accounts
• Not user-friendly formatting steps
• Steps disappear by a system issue

TesRail:

• Clearly functionality
• Not user-friendly formatting steps
• Not free edition
• BDD support

Testomatio

• New platform
• Can store manual and automation tests in one workplace
• JIRA integration
• BDD support
• Test automation Integrations

Choosing TMS depends on the project, needs, costs and etc. It works individually on each project

Defining Test Case life cycle – statuses

Defining life cycle – it’s also one of the parts of creating a good test management process. A lot of QA engineers. Many engineers do not know what to do with test case statuses. When there are statuses of test execution, this is understandable, but the statuses of test cases are completely different. Each project is considered differently: some use them, and some do not. Some systems do not have them at all. However, most importantly, work with statuses should be documented so that engineers do not have any questions.

In addition to the test cases themselves, it is necessary to think over the life cycle itself: to make it clear and simple for everyone. One of them:

Open → In progress → Test Execution → Done ✅

Sometimes, we are not changing anything and put in Open or Backlog and that’s all. However, test execution statuses are using always while testing with test cases:

• Unexecuted – test case has never been run before
• WIP – work in progress – QA engineer is working on that
• PASS – test case is executed without any issues
• FAIL – executed and some issues are found
• BLOCKED – test cases are blocked by another test case or issues

Defining Test Case Template and best practices/rules for your project

Depending on the project and team, the test case template may differ. One way or another, but the template must be there.

Therefore, for a large number of test cases, there will most likely be a large QA team. Even if the team will consist of only two members, the template will help to quickly write test cases and colleagues will be able to replace each other in writing test documentation. Thus, the most common pattern is:

• ID
• Summary
• Precondition
• Description
• Priority
• Steps
• Expected result
• Attachment

Priority of test cases is the main attribute for creating test cases if they are huge. In another case, the team should know which priority will be better for the appropriate test case. For this, need to arrange the meeting and clarify each attribute of the test case template

However, like requirements, test cases should have the characteristics of a good test case:

→ It is easy to understand and execute.
→ It is accurate with a specific objective.
→ It is easy to trace as per requirements.
→ It is repeatable, and can be used to test again and again.
→ It saves time and money by avoiding unnecessary steps.
→ It is reusable.

The maintenance process of test cases​

In this part, let’s clarify how to maintain a lot of test cases and not burn out. First, we need to understand if we need a lot of test cases. It is meant that it is necessary to think over the test case template, as well as how the test suites will be created, so that with the help of the minimum number of test cases it is possible to cover as much functionality as possible. After that need to identify if we can automate smth. It does not necessarily have to be a direct automation framework. These can be various tools that will help speed up testing. For example,

• Postman makes some automates tests for API
• FakeFiller for test inputting fields
• Different temp servers for verifying emails: https://yopmail.com/

Learn more testing tools you should use for Manual Testing:

• Top 15 Manual Testing Tools Checklist To Know

Even if there are many test cases, each test case must be linked to the requirements. To understand how the test cases covered the requirements. This will also help avoid the pesticide paradox, where a test case becomes irrelevant if requirements change. In addition, you can configure notification of requirements changes, and accordingly, this will be a signal for QA to check if the test cases are up-to-date. As a result, after each test run, you will get a report in any TMS system that helps you to analyze the results.

Also, don’t forget about the relationship between test cases. Actually, it’s bad practice to make relation cases, but sometimes it’s needed. For example, registration and login functionality. Without verifying registration, login checking is useless. For that, the case must be informed that these test cases are related between them. But, you can avoid this relation by writing Precondition. For example, for login verification – a user should be created. It doesn’t matter how we created this user. We are verifying login functionality. Using this method, we can reduce the related test cases and maintain them will be easier

😃 Let’s summarize and make a checklist for tracking thousands of test cases:

1. Identify TMS
2. Approve test case template
3. Create a test case according to the good characteristics of the test case
4. Make a test suite for different types of testing: smoke, functional, and regression
5. Identify and make some tests automated
6. Update or Remove test cases when it’s needed
7. Write a test guide of this on wiki project page

In addition, we suggest that you meet other points

Stackexchange discussion

• How to manage a lot of automated test cases

Reddit QA community’s opinion on how to handle a large number of tests on the scaled project:

👉 Join Reddit post by following the link