In today’s market, customer experience is king, and if your customers' needs are not met, the door opens for your competitor. From a business perspective, not only have enterprise-scale organizations been disrupted, but they are also generally not satisfied with their current software deployment speed.
With their sophisticated DevOps and continuous integration/continuous delivery (CI/CD) practices, companies such as Amazon and Netflix have raised the standards for more traditional businesses. As a result, many enterprises are on a quest to deliver quality software faster.
These organizations are moving toward DevOps—unifying development and operations with tools and best practices that ensure continuous delivery, continuous integration, continuous testing, continuous monitoring, and continuous feedback.
While these are important criteria for DevOps success, one ingredient is often missing: the measure of DevOps' business value. Here's how to approach it.
Measure the right things
How do you know DevOps is working? How do you know which customers are unhappy and why? Where are the bottlenecks, and why is delivery speed hampered? These are important questions, and there is a method to discern the answers. Without appropriate measurement, DevOps becomes more hype than value.
You may have seen the phrase "value stream mapping" pop up in DevOps discussions recently. Organizations have begun to adopt this lean measurement technique to ensure that the software they produce offers continuous improvements to the customer experience.
This way of looking at the software development lifecycle unlocks a variety of benefits: In a single pane of glass, you can collect and view critical data including value stream health status, performance of tool chain orchestration, tool chain activities/exceptions, DevOps reports and analytics, and traceability.
By managing the DevOps lifecycle with value stream metrics, managers can compare similar value streams across a common set of key performance indicators (KPIs) to determine whether DevOps efforts are successful. I've been having this conversation with industry leaders and analysts a lot recently, and I am seeing the message resonate with more and more business leaders and IT practitioners.
How to get started
Measurement will always look a bit different depending on your organization’s goals. Here are a few of the basic KPIs you should start with to help inform business decisions:
- Mean time between failures: The average time between when a problem is found in production and when it is fixed.
- Defect rate: The number of defects that are found during a given unit of time.
- Change lead time: The difference (in time) from when a change is submitted to when it is executed.
- Rework rate: The percentage of tickets mapped to releases.
- Unplanned work rate: The percentage of unplanned issues.
You can represent and share this data in different charts, use it to raise alerts or to trigger events elsewhere in the lifecycle. By ensuring that your team has a deeper understanding of the value and impact as information flows through a designated value stream, you can better identify where to focus future time and investments. Armed with this information, your team can make data-driven decisions.
Feel free to adjust the model to collect or present the data differently. If your team discovers a new metric to track, add that to the model. All the while, your system should continue to build up a data model that shows times, success rates, and other metrics important to your team. This "receipt" of your operations will come in handy when you need to demonstrate an audit trail across multiple, disparate tools.
DevOps is about visibility—and control
Anyone can do value stream mapping, but managing those value streams is a different ball game. To do it well, you must be able to see everything at once. Having visibility into every aspect of the DevOps lifecycle will make this infinitely easier.
Also, the ability to manipulate and control every process, event, tool, and so on from one place, rather than logging in and out of different platforms and rationalizing data from different sources, will simplify this process greatly.
As an industry, we need insights into the progress that the DevOps movement has brought to our delivery processes and tools, and we need the power to tweak those processes for more optimal results based on data and data science.
Now step back and reassess
In this next stage of DevOps, take a step back and follow the value. Start with a bird's-eye view, and then zoom in and out as needed. If you're focusing on release automation alone, you're not thinking big enough.
DevOps measurement and analysis will move front and center this year. As the saying goes, "You can’t manage what you can't measure." We'll never know if our DevOps efforts are successful without using value streams to evaluate those efforts and make smarter decisions.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
