The cost of change curve is a prevalent waterfall illustration for explaining the importance of up-front planning and design. The basic notion is that change becomes more expensive the longer you wait. If you change your mind while planning, it’s as simple as changing the plan. If you change your mind while analyzing requirements, all you need to redo is analysis. If you make changes later in the process—especially once code is running in production—the cost of change is around 10 times as high. This is because you need to plan again, analyze and design again, develop again, test again, and deploy again.
If you’re familiar with agile software development methodologies, you’ll recognize that our industry is moving away from this way of thinking. Kent Beck and others argued that Extreme Programing (XP) practices could help flatten this change curve by enabling change—no matter how late in the process. These practices are quite widely adopted now and have helped many organizations pivot their products and services when required—without incurring the exponential costs described by the cost of change curve.
The list of XP rules only makes two mentions of the release process itself, and it doesn’t say anything about how to approach the software once it’s released, other than stress the value of the fast feedback provided by small, frequent releases.
DevOps practices can push the boundaries further, reducing the cost of change dramatically. These practices tend to focus on what happens while and after software is released, and each of them makes it cheaper to make changes late and often. They can even help you deal with what happens when things go wrong, as they so often do.
Here are five ways to cheat the change curve with DevOps.
1. Reduce deployment friction with automated deployments
XP taught us that small releases lead to fewer surprises in production and make it easier to isolate changes that cause issues. XP also advocates that we release frequently, but the reality is that reliable, frequent releases are not possible without automation.
Automated deployments (often referred to as push-button or single-click deployments) are the cornerstone upon which many other DevOps practices are built. Every manual incantation you automate will decrease how much it costs to get a simple change live, because you don’t need to know everything about a complex deployment process. This frees up experienced team members to focus on more valuable tasks, rather than helping trivial changes get to production.
2. Reduce knowledge silos with infrastructure as code (IaC)
The DevOps movement has ushered us away from the era of treating servers like members of the family. "Infrastructure as code" is a phrase used to refer to the practice of configuring servers using code, rather than manual steps taken by a person logged into the server. This has been liberating in two ways.
First, the server is now an open book that anyone can read and understand. We no longer have to spend days guessing which setting is causing a server to behave in a certain way. Instead, every step of the server’s configuration is laid out in code, creating an auditable, accessible record of what happens when we create, or deploy to, a server. It makes it cheaper for a new team member to learn what it takes to maintain, or deploy to, your server when necessary, because you’ve documented the process with code.
Second, it frees up the person who used to spend all of his or her time keeping the server running. This means that person can now share his or her valuable knowledge of how to run servers with the rest of the team. It also means that person is no longer a bottleneck for changes.
3. Reduce recovery time with phoenix servers
What will you do when the worst happens, or when you make a mistake that causes your whole server to go down? Downtime is not acceptable to most customers. The term "phoenix server" refers to a server that you can rebuild from scratch in the event of such a catastrophe. In fact, in order to make sure you’re ready, you should throw away your server almost as often as you release changes. Immutable servers are those you rebuild every time.
This approach will clearly save money in the event of disaster, but it’s not immediately obvious how this reduces the cost for each individual change. The answer is that it reduces the risk of each change. The more robust and well practiced your recovery strategy is, the more comfortable developers will feel taking changes to production. This means that features, and bug fixes, reach your customers faster.
4. Reduce complexity-per-change with microservices
Code is complex. In any codebase, it takes time for someone to learn enough about the code to make a change. For monolithic codebases, this can often take a very long time. Adopting a microservice architecture means the code related to a specific change will be much narrower in scope, and a developer will be able to learn what he or she needs to make the change much faster. (Things do become more complex at the boundaries between services, but the increased complexity is more clearly defined and easier to notice.) In a small-enough service, a developer brand new to the codebase could make a critical change in a matter of hours, rather than days.
5. Reduce uncertainty with QA in production
For many teams, quality assurance is a process that stops just before code goes live. With QA in production techniques, you can make sure you keep learning about the quality of your features after they go live. They can guide the changes you make next in a way that an analysis or design document from months ago never could—because it’s based on what is happening right now, while real users are using your system.
Monitoring in production (combined with the small, frequent releases that XP promotes) will help you assess the impact of the changes you release to production. This could be a negative impact (e.g., a bug introduced in the last change that reduced performance for mobile users) but could also be a positive impact (e.g., a new feature that has increased sales or page views).
Invest in DevOps skills
Change is inevitable, and those organizations that fail to adapt will quickly falter. If building software is core to your business, as it increasingly is for many, it’s important that your tech can be changed to respond to the quickly changing consumer market. Reducing the cost of that change is vital. Investing in the right DevOps practices pays off in the long run.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
