It can be difficult to pick a test framework to use on a project. You have so many options available that it can be overwhelming to decide on the one that works best for testing a particular application.
So how do you decide? Here's a step-by-step guide on how to choose the optimal test framework for any given type of testing.
Understand your testing requirements
First things first. Don’t jump into tooling or try to figure out how to solve a problem without being clear on what the problem is. It's critical to understand the testing requirements and the scope of testing; only then should you start to make a plan on how to build the framework.
I recommend answering the following three questions at the beginning of this journey, and then coming back to them at various points later on. It can help with validating that the right areas are being tested and that there are good processes in place for implementation and maintenance.
Which are the most important areas to test?
Determine what you want to focus on before jumping into testing. Identify the highest-priority application features or areas that must be tested.
Identify a list of scenarios within each feature to test, and to consider which level of testing can fulfill each. It helps to understand the foundations of test automation and to identify the most important areas to test, as well as at what levels you should test them.
Who will own the framework?
Discuss which individuals or groups will own the test framework. It's important to do this to make decisions going forward about how the framework should be structured.
In my experience, this works best when a whole group shares the responsibility and vision of the framework to work together on it over time.
How will the framework be implemented and maintained?
After you determine who the owners or stakeholders are, create a high-level plan for how the framework will be implemented. Think about the steps you need to take to get the framework set up, and then consider the scope and the timeline of the project.
After implementation, there will be other tasks, such as adding new tests and maintaining the framework. Discuss with the stakeholders when tests should be added in the feature development process and if there is a certain level of test coverage you should stay above.
Research frameworks
The next step of this journey is to figure out what it is you want in a framework. Follow the three steps listed below:
Search for frameworks by type and language
There are three main levels of testing: integration, unit, and functional. And there are even more types of testing within these levels. Each level is different and might have different frameworks available.
In terms of a programming language, it makes the most sense to write tests in the same language used to create the application. This way engineers working on the test framework should be able to get up to speed quickly and understand how the framework works.
Pick one level of testing you want to focus on and the programming language you want to use. An example of this is to search for a JavaScript unit testing framework. From the search results, make a list of five to ten frameworks that fit your criteria.
Look for the pros and cons in each framework
The first thing I look for in a framework is that it be straightforward to implement and build upon.
I'm also interested in seeing how well the framework is supported in terms of documentation and customer support. And it helps to know if the framework is actively being worked on, as well as if it is widely used in the tech community.
I also want to know whether the framework is open-source or paid. There are plenty of both types, but whenever possible I choose open-source frameworks. If your organization is willing to pay for a framework, work with management to decide how much is a reasonable amount.
When determining the pros and cons of a framework, don't be limited to the factors listed above. Feel free to pick your own factors, and for each framework in your list, decide if the criteria are considered pros or cons.
3. Try out potential frameworks
I recommend doing a spike of each potential framework on your list. An example of a spike would be taking a few hours to write a couple of tests using a specific framework. The goal is to get a better understanding of how each framework works so that you can make an informed decision about whether a given framework will be a good fit.
Make the choice as a team
After researching frameworks, determining pros and cons, and spiking a few options, you should have knowledge of each framework on your list and be able to rank them. Have the stakeholders get together to discuss each option.
After the discussion, your team will likely have found at least one framework that seems like the optimal choice. Select one, fill in details in the plan on how to implement the framework, and then go for it.
If there are no frameworks that match your criteria, consider building your own test framework. This isn't the route I'd recommend, because it's challenging and time-consuming, especially in the beginning. But if you decide this is your path, you can follow this guide for advice on how to do so successfully.
It’s also okay to change your mind later. Things change. Applications change. What you choose today may not be the best choice a year from now. This is not ideal, but it is realistic. I’ve been on projects where we decided to change the test framework just a year after it was implemented, and others after five or more years. While it will be time-consuming to make a big change, it will end up for the best.
Easy pickings can be yours
So there’s my step-by-step process for choosing the testing framework that will work best for your application. The goal is to first identify what needs to be tested and then investigate to determine which framework will best support that testing. Following these steps will make it easy to pick a framework for any type of testing, anytime.
Share your thoughts on how to choose test frameworks in the comments below.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
