As the world has moved toward more automation, so has software testing. But if you run automated tests only at the end of your development cycle, you won't reveal all the possible issues your customers may face. Continuous testing is a great way to incorporate a wide range of testing methods early into the software development cycle in order to catch defects that could cost the business money.
How early? People in the tech space often think that continuous testing means running automated tests during the development phase, not just at the end of it. But it is much more than that.
Continuous testing is the act of testing an application at every stage of the lifecycle. That means incorporating testing from the initial conception of an idea to its release—and beyond. Here's how it can supercharge your software development.
The clear benefits of continuous testing
So, what are the benefits of true continuous testing?
Consider this graph from Capers Jones, American software engineer and author of the book Applied Software Management: Global Analysis of Productivity and Quality.
Jones’ research shows that upwards of 85% of bugs are introduced in the design and build phases of development. That's often at the point where engineers haven't written a single line of code.
A bug found post-release could cost $16,000 or more to address. But developers could have fixed that same bug in the early stages for as little as $25. It's abundantly clear that the earlier you identify and act on an issue, the less financial impact it will have on your business.
If you incorporate testing early in your design process, you will catch the pesky bugs that could cost you in time and money later on. That's why continuous testing is crucial to improving the quality of your product.
Putting continuous testing into practice
Testing can be done at every stage of the development process, from planning to coding to release. Take test-driven development, for example. This is the concept of writing code first that is set to fail a test, then writing the code to pass it. It puts testing at the forefront of the design process, even before any code is written.
Dan Ashby, head of quality engineering at Photobox, perfectly articulated how you should approach continuous testing in the diagram below.
It is possible to test at every stage of the process. Source: Dan Ashby.
Despite agreement by many QA experts about testing at every stage, people often ask me, "Surely it's not worth testing at every stage of the development process?"
Yes, it is!
Case in point: Blackboard
Consider the case of Blackboard, an educational technology company that uses a continuous deployment methodology. The nature of its product release strategy means that continuous testing is of the utmost importance.
When Owais Peer and I spoke with Ashley Hunsberger, director of release engineering at Blackboard, for our book, Leading Quality, we discussed the value of continuous testing. Hunsberger described what led them to turn around their testing strategy:
"While we had people who believed in a quality culture, it still fell on our testers to be responsible for it. We asked ourselves, 'How can we speed up delivery, be more productive, make our lives easier, and move our philosophy of quality from playing a big part to being the foundation of our culture?'"
Mapping dev stages to testing types
Hunsberger's team mapped out the development pipeline together, detailing each stage of the process from first code commit to production deployment. Once the team was clear on the process, it matched every stage of development with the most appropriate testing type.
For example, just after merging the feature branch into develop, the team needed to ensure that it hadn't accidentally introduced any bugs into the codebase. Hunsberger's team analyzed appropriate testing methods and decided that continuous integration tests were the most effective method of testing for this specific step.
In this way, the team only implemented testing techniques that were well-suited to what it actually wanted to test. Rather than use one method right before deployment, it meticulously planned how to best test each step.
In this way, the team dramatically reduced its iteration loop. Once it started finding and addressing issues faster, the team saw a considerable reduction in the number of critical problems making their way into production.
Make continuous testing part of your QA strategy
The Blackboard story shows the tangible value of continuous testing. The team reduced the number of critical issues in production, simply by taking a deep dive into the development process and creating an airtight continuous testing strategy.
It's a well-known fact that 62% of mobile users will delete an app outright if they find a bug in its functionality. So, by delivering a bug-free app, you can ultimately prevent a loss in revenue, and increase your growth. With continuous testing, you can achieve this by testing efficiently at every stage of the development process.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
