No one needs to tell you that test automation will improve quality and save your software development organization from spending unnecessary time and expense on manual regression testing. You've heard that a million times.
What you need to know is how to get there—and avoiding common test automation mistakes.
Rather than spending weeks at the end of the development cycle going through a hardening phase, you want to run automated tests that take a fraction of the time and run regression tests with each build. Unfortunately, many organizations start at the user interface layer, which delivers the smallest return on investment. This is where Mike Cohn's test automation pyramid concept can help. Follow this guide to get the most bang for your buck as you get started with test automation.
Using the test automation pyramid
Don't start by running out and hiring a bunch of contractors to automate at the user interface level. Instead, spend time familiarizing yourself with the test automation pyramid. Then, when you're ready, create a test automation strategy that will optimize your ROI.
The test automation pyramid, first introduced by Cohn in Succeeding with Agile, shows how you should maximize automation, starting with your unit tests at the lowest level of the pyramid and moving on to service level testing. User interface testing sits at the very top. Unit tests are fast and reliable. The service layer allows for testing business logic at the API or service level, where you're not encumbered by the user interface (UI). The higher the level, the slower and more brittle testing becomes. Finally, while some UI test automation should be done, such tests are slower, more difficult to maintain, and break more easily. Keep those to a minimum.
The foundation: Unit test automation
The unit test is an important part of writing high-quality code. When people in software organizations speak of test automation, they tend to think of tools such as Unified Functional Testing (UFT) or Selenium, which provide test automation frameworks. However, developers should write the majority of automation tests at the unit test level.
Developers can use unit test frameworks such as xUnit or Microsoft's Visual Studio Unit Testing Framework to create automated tests for small units of code. Some agile teams use test-driven development, a technique in which you write the unit test before the code to help drive code design. Some developers write the code first, but don't consider the code complete until they've developed an associated automated unit test. You can assess whether each code path has been tested with test a coverage tool such as DotCover.
Automated unit tests are extremely fast to execute, and you'll want to run them after every build. This approach will give your team immediate feedback when regressions occur, as your code base continues to grow and evolve. Because the tests are so small and specific, it's easy to troubleshoot them when you have a failure. Having these tests gives your development team the peace of mind to refactor with confidence, safe in the knowledge that they'll quickly detect any new code that causes regressions.
The middle layer: At your service
Cohn refers to the middle layer of the pyramid as the service layer, but it's also known as the layer for automated API tests, automated component tests, or acceptance tests. You use this automation layer to test the business logic without involving the user interface (UI). By testing outside the UI, you can test the inputs and outputs of the APIs or services without all the complications the UI introduces.
Testing at this level gives your testers the option to set up data and go through a series of tests with the inputs and expected outputs you've defined in separate spreadsheets or files. This lets your team create automated tests against boundary conditions, edge cases, or error conditions, without involving the UI. These tests are slower and more complicated than unit tests because they may need to access a database or other components. You should absolutely use them, however, as they're still much faster and more reliable than UI tests.
The cherry on top: The UI layer
Congratulations: now that the majority of your code and business logic has been tested, most testing at the UI level has been eliminated. Your focus now at the UI level is simply to ensure that the UI itself is working correctly. UI tests are very brittle, so keep these tests to a minimum. These automation tests will need maintenance any time the UI changes, and because there are so many factors that come into play when you run a test that emulates clicks on a screen (such as network speed), such tests can result in false test failures. You can't ignore those test failures, but you don't want to end up spending more time troubleshooting and maintaining UI tests than you spend finding actual code defects.
With strong automation test design, automated UI tests will round out your automation test suite nicely. If you need more help here, I recommend Chris McMahon's article, Automation testing: Seven tips for functional design. In it, McMahon offers technical advice for creating powerful and maintainable UI automation tests.
Take the test automation pyramid diagram and put it on your wall. It should serve as a reminder that the majority of automation tests should be at the unit test level, followed by those that can be executed at the API or service level. Finally, with strong test design, you can write a minimum set of automated UI tests to complete your automation test suite. Once you have this solid set of automation tests at your disposal, regression testing will be a breeze.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
