The Accelerate: State of DevOps 2018: Strategies for a New Economy report has arrived, and it's got more than a few surprises in store. Among other things: Your cloud implementation is most likely wrong, and outsourcing is not your friend when it comes to performance.
Produced since 2012, the State of DevOps report evaluates software development and delivery practices in a wide variety of industries. It looks at platforms, practices, culture, tools, and outsourcing methods to determine the characteristics common to high-performing—as well as low-performing—software development and delivery teams.
The result is a comprehensive guide not only to the current state of DevOps, but also to what DevOps professionals can expect to see in the near future. As usual, the State of DevOps report this year contains fresh insights. Here are seven key takeaways.
1. Many DevOps shops believe they have implemented a cloud model but have not
Nicole Forsgren, chief scientist for DevOps Research and Assessment (DORA) and the chief author of the report, said that one of the most significant and surprising findings in the 2018 report was that "many software development teams think they're doing cloud, but they aren't doing it right."
Nearly a quarter (22%) of respondents said they had built a cloud model within their enterprise, but upon closer investigation, their models did not include at least one of the five requirements for cloud computing. These are:
- On-demand self-service: Customers can provision computing resources as needed, automatically, without any human interaction required.
- Broad network access: Capabilities are widely available and can be accessed through heterogeneous platforms.
- Resource pooling: Provider resources are pooled in a multi-tenant model, with physical and virtual resources dynamically assigned and reassigned on demand.
- Rapid elasticity: Capabilities can be rapidly provisioned and released to quickly scale outward or inward, commensurate with demand.
- Measured service: Cloud systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate for the type of service.
"When customers tell me they have implemented cloud but it isn't working for them. I tell them we need to take a closer look at their model. It almost always fails to meet at least one of the requirements."
—Nicole Forsgren
2. Good software-delivery performance boosts organizational performance
For the 2018 report, DORA added availability to its development model to create a new metric called "Service Delivery and Operational (SDO) performance." DORA describes availability as "an ability for technology teams and organizations to make and keep promises to, and assertions about, the software product or service they are operating."
In other words, it's a measure of whether software delivery teams are making sure their end users can always use their product.
DORA's analysis shows that high availability is closely correlated with high software-delivery performance, with its top two levels of performers having the best availability practices. The report further found that having strong SDO metrics boosted the performance and productivity of the organization overall. Ensuring that software is delivered in a timely and usable manner in turn meant that the organization could run as smoothly and efficiently as possible.
3. Open-source software improves software-delivery performance
While more than half (58%) of respondents said they "make extensive use" of open-source components, libraries, and platforms, the most extensive users of open source were the highest-level performers. The study found that open-source software is 1.75 times more likely to be extensively used by the highest-level performers, and these users are also 1.5 times more likely to expand open-source usage later on.
4. Outsourcing impedes performance
While outsourcing has been embraced as a fast and low-cost way to expand an organization's technical workforce, "it comes with extremely high costs," said Forsgren. The study found that outsourcing software delivery resulted in higher lead times due to batching jobs and that all features in an outsourced application, regardless of their value, tended to be delivered at the same speed.
These effects hurt performance, since long lead times quickly led to backlogs. Delaying high-value features also slowed down the organization and was found to alienate customers.
High performers limit outsourcing as a matter of course. In fact, elite performers, the most effective level of developers in the study, rarely outsource by function.
While outsourcing can save money and provide a flexible labor pool, low-performing teams were found to be almost four times as likely to outsource entire functions, such as testing or operations, as their highest-performing counterparts.
"A lot of times outsourcing creates silos within silos."
—Nicole Forsgren
She added that these silos, and multiple handoff points, ultimately reduce performance and make the true costs of outsourcing exceed any benefits from the flexible labor it may provide.
5. Good technical practices drive high performance
Key development practices such as monitoring and observability, continuous testing, database change management, and security that's integrated early in the software development process all increased software delivery performance and overall organization performance.
"Good deployment techniques reduce burnout."
—Nicole Forsgren
This performance extended to product management practices such as lean product management tenets, including team experimentation and working in small batches with frequent releases. Organizations that follow these practices had the same positive effects through improved organization culture and staff productivity.
Critically, shifting left on security promoted success in continuous software delivery. "Doing the security review at the very end of the development cycle often slows delivery down a lot," said Forsgren. High performers found that incorporating security into the earliest stages of development could solve security issues when they were smaller and easy to fix.
6. Industry is irrelevant to achieving high performance for software delivery
Don't feel bad if you’re not in a cutting-edge industry. You can still achieve success.
Top performers were found in both nonregulated industries such as manufacturing and retail and in highly regulated industries such as finance and transportation.
7. Culture is a key driver of DevOps performance
Working with sociologist Ron Westrum, the researchers at DORA found that cultural attributes such as autonomy, shared risk, a tendency to break down silos, and a climate of learning not only accelerated software delivery, but also boosted organizational performance as a whole.
That may be the trickiest shift of all in the report, however. As one anonymous developer said about this issue, "Cultural impact is interesting. Too bad it's nearly impossible to change."
Noting that the report is commonly used by developers anxious to compare their level of success with others, Forsgren said that comparison is really just an entry point.
"'How are we doing?' is usually the first reason people turn to the report. Once they see that, we have their interest and can talk about how they can improve."
For more on the state of DevOps, get the Accelerate: State of DevOps 2018: Strategies for a New Economy report, and see Nicole Forsgren's presentation at the upcoming DevOps Enterprise Summit, to be held in Las Vegas October 22-24.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
