DevOps is fueled in large part by feedback. The software development evolution—from old-school waterfall methods to agile development—gets applied with DevOps on a broader scale, and to the whole organization. One of the defining characteristics of agile and DevOps is frequent adjustments and course corrections based on continuous assessment. But to maximize efficiency with DevOps, teams need to have processes and tools in place to provide that feedback information.
The goal for most teams of developers is to increase visibility into customer usage. Continuous assessment helps organizations:
- Monitor and measure everything
- Create constant feedback loops
- Make course corrections during ongoing projects
- Prioritize business, security, and dev planning
- Iterate more effectively
These capabilities are improved through visibility across the different teams involved. But in most cases, developers don’t have the end-to-end visibility they need, including real-time dashboards and insight at the code level. Developers often turn to IT admins in search of relevant metrics and data, but the result is typically mixed and generally inadequate.
Continuous assessment
Much of what DevOps needs is encompassed in continuous assessment. Continuous assessment provides the consistent feedback necessary to fuel the rest of the DevOps components. So, let’s examine what components are important for success with continuous assessment, and how organizations can drive innovation that is both achievable and measurable.
Colin Campbell, director of technical content at Chef, explains, “The basis of continuous assessment is automation. Automation covers many areas. For example, the key to an effective DevOps workflow is automating the configuration and management of development and production servers, which provides significant data inputs for transparent and ongoing assessment.”
Skilled DevOps practitioners automate the workflow itself using continuous integration and delivery techniques, which allows for monitoring and assessment to occur at every stage of the pipeline, from initial check-in by a developer to monitoring system behaviors seen in production.
“So, by first instituting basic configuration management automation and then using this foundation to create a continuous delivery pipeline for infrastructure and application change management, organizations can constantly monitor and assess the state of their systems as they simultaneously increase rate of development,” says Campbell.
3 steps to measurable innovation
High-velocity innovation comes down to three primary steps: Understand what customers want, define achievable goals, and make sure progress is measurable.
Step 1: Know what they really want
One of the most important aspects of DevOps and continuous assessment is making sure that the end result meets the needs of the customer. Nicole Forsgren, director of organizational performance and analytics at Chef, stresses that innovation starts with a customer focus—whether the customer is internal or external. She strives to ensure that her team is creating the thing that's actually needed.
“Ignore HiPPO (Highest Paid Person in the Organization), in the words of author Jez Humble] and go straight to the source,” says Forsgren. “So find out from your customers what they want. Find clever, innovative ways to experiment and identify the end goal. Once the goal is understood, make sure this goal is clearly articulated and well understood throughout the team and organization. As I'm fond of saying, words have power, and definitions matter.”
Step 2: Make sure you can get there
Once the end goal is understood, the next challenge is to ensure it’s achievable. Teams need to iterate frequently and focus on small interim steps. Developing in smaller increments makes it easier to monitor and drive progress, and it allows for regular feedback and adjustments to make sure everything is on track. Periodically, often monthly, the team should review the end goal and come up with interim goals for the month. At the end of the month, the team should review its progress and define interim goals for the next period, again with the end goal in mind.
Ultimately, progress has to be measurable and fueled by direct feedback from customers and real-time usage analytics. Forsgren says, “To make sure the work is measurable, the identification and definition of metrics are very important. Again, words have power, and definitions matter. This is important, because metrics can also serve as a way to communicate and foster alignment across teams and throughout the organization.”
Step 3: Measure your success
With a customer-centric focus and the end goal in mind, teams need to identify a measure of success—some external metric or outcome. From there, it’s a matter of working backward. Consider the internal actions or processes that need to occur to help drive the desired outcome. Once an organization has determined those actions or processes, it can identify the metrics that represent a proxy for those actions, giving the organization a visible metric that can be used to monitor and measure success.
Once you have the capabilities, exploratory data analysis can be powerful as well, but chasing data down rabbit holes is never a good place to start. “This is important because all too often I see organizations capturing all the metrics they can easily capture, without stopping to think if they should impact the outcome measure," Forsgren says. "There will always be limitations to your data collection efforts, because not every action or effort will have a perfect data proxy. But measurement is also something that can be iterated and improved upon—both in proxies. (i.e., is there another measure that captures the underlying idea better?) and in drivers (i.e., are there other drivers that are more effective?).”
You need all of these elements working in harmony, though. Setting achievable goals and measurable outcomes that don’t meet customers’ needs is a waste of time, while understanding customer needs but setting goals that are not achievable is a recipe for failure. You must have some measurable way of determining if those goals have been met.
Workflow matters
When you combine all three steps noted above, the result will almost certainly result in some improvement. Organizations that understand what customers want and need, and then establish a workflow that stays focused on those needs with achievable goals and measurable outcomes will succeed every time.
Photo: Thomas Rousing/Flickr
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
