While DevOps may be overhyped, oversold, and oversubscribed right now, those who approach it as a journey, not a destination, are finding incredible business value. You’ve heard all the reasons why DevOps won’t work: issues with compliance, ITIL, security, production availability, architectural complexity, and so on. But the benefits it can deliver—faster delivery, quality, and security—are within reach. The question is, How do you start the move from silos to streamlined development pipelines?
First, be sure you have the definition of DevOps straight. If you’re not clear about exactly what DevOps is, check out Gene Kim’s personal definition, Eric Minick’s summary of DevOps definitions, and IT Skeptic blogger Rob England’s exploration of DevOps definitions.
Recently, IT leader and author Gary Gruver shared his agile and DevOps leadership experience with me. “Building a delivery pipeline can be the forcing function that drives the DevOps transformation, one small change at a time,” he said. He was talking about optimizing the value stream of a business idea, moving it through IT to customers to deliver business value. That's old news to manufacturing, but now it is fueling this IT transformation we call DevOps.
Moving from silos to pipelines
As IT systems have increased in complexity and in value to the business over the years, IT has adopted a principle of specialization and focus in which teams develop best practices and optimize their work. In a factory assembly line, specialized equipment (molds, dies, stamps, etc.) perform specific functions. IT silos are similar, but they can also be self-serving. Because silos typically measure what they can control, too often they are rewarded not for their business value but for meeting an internal key performance indicator. To understand how a silo’s myopic view can be harmful to the business, read Eliyahu M. Goldratt’s “The Goal.”
The business demands continuous delivery of value, and that value is created only when a product or service is delivered to a satisfied customer. It’s not created when one silo or step in the process is completed, but only when the customer is satisfied. This means that you need to reset your focus from silos to the end-to-end flow of value from business idea to customer delivery (production). You can visualize the flow of value as a pipeline. Or, if you don’t like the word “pipeline,” then choose your metaphor. Assembly line, value stream, product, or service—they all work.
Why a pipeline?
Imagine this physical example: You have a fleet of oil tanker trucks, each of which carries a huge load of oil. These trucks navigate icy, mountainous roads and frequently stop for inspections and repairs. The fact that these trucks sometimes get delayed in traffic or have accidents is a huge challenge to maintaining predictable oil delivery. You need to schedule deliveries, but it’s difficult due to all of the variables. Now consider the oil transportation pipeline. You can move small amounts though the pipeline fast. That removes complexity, reduces risk, and enables management and improved flow. The pipeline has specific scope (system boundaries) and a specific beginning and end.
The same idea applies to IT. Rather than oil, an IT pipeline delivers changes, such as quarterly software releases. You probably already have some sort of a pipeline in your own IT shop. But is it well defined, or is it full of leaks, cracks, twists, and obstacles? Traditional IT processes periodically deliver many related changes all at once, essentially packing everything into several big delivery trucks. It’s not fast, it's not efficient, and it's not easy to test. Big changes mean high risk.
In contrast, a delivery pipeline enables the flow of smaller changes, with a focus on flow. Your teams can concentrate on optimizing the delivery of changes that bring quantifiable value to the business. This approach leads teams to continuously monitor and learn where they are encountering obstacles, resolve those issues, and gradually improve the flow of the pipeline. As the process continues, the feedback loop provides new insights into new issues and obstacles to be resolved. The pipeline is the focus of your continuous improvement loop.
Where to start your DevOps journey
Actually, where you start isn’t the real question. Deciding to start is the first step in a process of many incremental improvements. Scope is the second step—determining the boundaries of what is and is not included.
In deciding on scope, you need to view the problem from the perspective of the value of what's being delivered at the end of the pipeline, as opposed to a subset of the pipeline. Many great tools are available to help you define scope. Customer SIPOC diagrams (suppliers, inputs, process, outputs), for example, help to clarify the inputs, outputs, and customers of your pipeline.
Now you're ready to start. But should your team begin by automating the build process and continuous integration (CI)? Or should you start with infrastructure as code and automate the provisioning of environments and systems with continuous delivery (CD)? Or perhaps you should start with automated testing to ensure quality, security, and stability of each change in the pipeline? Yes, yes, and yes. Any of these is a perfectly good place to start, assuming that your team understands the scope and objectives. Specifically, your team must:
- Own its pipeline and share a common goal
- Work to continuously improve the pipeline
Exactly where your team starts should be left up to them, but they will need high-quality feedback through continuous assessment of the pipeline and system so they know what’s working and what’s broken. Following this iterative improvement approach is the DevOps journey. For guidance, see Deming’s Plan Do Check Act (PDCA) continuous improvement methodology and John Boyd’s Observe, Orient, Decide, Act decision cycle loop (OODA).
Who has to change?
DevOps can't work without organizational changes. These must come in three areas of responsibility:
- Executives and managers. While DevOps initiatives can start as a grass-roots effort, real transformation comes from leadership. Executive leadership creates the ecosystem of goals and roles, and leaders empower their teams. The most important thing leaders can do is create an environment where their teams can experiment, learn, and improve. As a leader, you model the behavior and set the vision, and you should start by educating yourself. Good resources for managers include The Phoenix Project, Leading the Transformation: Applying Agile and DevOps Principles at Scale, The Goal: A Process of Ongoing Improvement, and Lean Enterprise: How High Performance Organizations Innovate at Scale. Also, consider attending the DevOps Enterprise Summit later this month.
- Teams. Rather than thinking about how each silo needs to change, your team should focus on the fundamental issue, which is that they must have clear ownership of the pipeline. This is where Werner Vogel's concept of “You build it, you run it” becomes critical. Teams (development, test, and operations), need to converge around the ownership and operation of their pipeline. For team learning, I recommend reading Phoenix Project, The Goal, and Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation, as well as attending DevOps conferences on this list, following the top DevOps leaders on Twitter, and checking out these 53 DevOps resources.
- Governance. Governance teams (program management office, etc.) need to evolve to support a different time horizon. Rather than focusing on projects and discreet business cases where there is a clear return on investment, they need to adopt a longer-term perspective, since the measure of success extends to many releases over a much longer time horizon.
So what are you waiting for? More responsive IT, higher quality, and improved security are within reach. Start the journey now. Your competition probably already has.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
