Rewrites vs. refactoring: 17 essential reads for developers
Refactoring is more about keeping things simple and flexible than it is about getting things right. Getting things right often involves adding new capabilities, or redesigning large sections of an app. Keeping it flexible just makes it that much easier. Thus, refactoring is best seen as an enabling activity.
The conversation around refactoring vs. rewrites has been going on for decades. While it's certainly a timeless topic about the way we design and maintain our code, the accumulated knowledge around refactoring is constantly being rediscovered by new developers, leading to recycled conversations and debates in many cases.
Case in point: A blog post a few months ago called "Rewriting from scratch, should you do it?"
The conclusion in that article is that a rewrite is ultimately the better solution. This goes against a lot of the literature around refactoring vs. rewrites, which almost always favors restructuring and refactoring. But many of those resources were written for large systems, not small utilities such as the one referred to in that blog post (Info-Zip).
Many of the old but still essential resources on refactoring were also written before the emergence of the microservices trend, although service-oriented architectures have certainly been around for much longer.
Here are some resources (old and new) about refactoring that will refresh our memories about the long-accepted rules and ideas around refactoring and rewrites, and hopefully restart some thoughful conversations.
1. Martin Fowler's treasure trove of refactoring guidance
Any conversation about refactoring should start with Martin Fowler and Kent Beck's book, Refactoring: Improving the Design of Existing Code, which popularized the term and the modern ideas behind the practice.
"Improving the design of existing code." That's a theme that's strongly repeated throughout the resources in this article. Just because you made bad design decisions when you first started building a system (or maybe they made sense at the time, but conditions have changed), that doesn't mean you have to stick with what you have. Designs can be changed and improved, if you do it carefully. And often, that's the smarter decision.
One of the biggest problems that still persists in many developers' perception of refactoring is the rampant misuse of the word.
Refactoring means little changes that specifically make the code easier to read and modify. If it changes the behavior of the application from a user perspective, then it isn't refactoring. Often, developers say they're refactoring when they're really restructuring. And because that is also important for maintenance and improving the design of your code, here are some good reads that cover that as well. But these days, many developers lump restructuring under the term "refactoring," so be aware.
There are dozens of additional articles and resources on refactoring from Fowler's blog (Refactoring Pipelines is a newer post that you should check out), but most of those pieces are available through this compilation site: Refactoring.com
2. Refactoring Patterns, by Joshua Kerievsky
A few years after Fowler and Beck's watershed refactoring book, Joshua Kerievsky wrote a book introducing pattern-directed refactorings. It's an important read after Fowler because it provides lots of tactical guidance for refactoring that you'll definitely need.
Knowledge of the Builder pattern once played a crucial role in the evolution of a system I helped develop. The system needed to run in two completely different environments. Had we not evolved the system to use the Builder pattern relatively early in our design, I shudder to think of how our design would have turned out.
—Joshua Kerievsky, Refactoring to Patterns
3. Refactoring is untangling
"Untangling" is an interesting analogy from a blog post by Cody Powell. It's a good, fairly recent reminder of the evidence that shows us why refactoring needs to be a small, gradual activity. If you try to make refactoring bigger than the Fowler definition you run into problems.
If you proceed with a big refactor, you'll trade one form of technical debt for another.
In order to truly pay this debt down, you need to untangle the complexity around the problem. You need to spend time looking at all the clients calling this component. You need to spend time talking with your colleagues, learning more about the component's history and how it's used. You need to make a few simplifying changes around the periphery of the component and see what works. Each week, you spend a little more time and untangle the problem just a little bit more. Given a long enough timeframe, you'll eventually untangle all of the complexity and brought a teeny bit of order to the universe.
—Cody Powell, It's Not Refactoring, It's Untangling
4. Code Simplicity
Code Simplicity, by Max Kanat-Alexander, is another major milestone in the refactoring conversation. Writing in 2012, Kanat-Alexander draws on his experience as the former lead developer of Bugzilla—which he saved from a highly complex codebase when he first started working on it in 2004—and from his experience as the technical lead of engineering practices at Google's YouTube division.
One of the key themes of the book is that, although rewriting is always a very attractive option to many developers, refactoring and restructuring the existing codebase is almost always the better option. You'll have to read the book to understand how strong his case for this argument is, but below I'll give you the extremely summarized version of his five criteria for when rewriting is acceptable. It gives you an idea of how strongly you should consider refactoring over rewriting.
Only rewrite a system if all of the following are true:
You've done extensive research and you're nearly certain it will be less expensive and less time-consuming to rewrite.
You have a ton of time to devote to building the new system.
You're a better designer than the system's original architect. If you are the system's original architect, are you sure you're significantly better now?
You've got a plan to get feedback from users early and often as you rewrite.
You're rich enough and have enough time to maintain both systems while you build the new one.
5. Joel Spolsky's 'Things You Should Never Do, Part I'
StackOverflow founder Joel Spolsky made strong arguments against rewrites 12 years before Kanat-Alexander's book in a blog post: Things You Should Never Do, Part I. So you can see that there's a clear line of refactor-over-rewrite support from the late '90s all the way to 2012, coming from the biggest luminaries in software development.
Spolsky writes that there are three main problems developers have in their code when they want to rewrite it:
- Architectural problems
- Inefficient code
- Hard-to-read, hard-to-maintain code
And his solutions to these are:
- Fairly major architectural changes can be done without throwing away the code.
- You don't have to rewrite the whole thing. When optimizing for speed, 1% of the work gets you 99% of the bang.
- Frankly, this is the kind of thing [making the code easier to read] you solve in five minutes with a macro in Emacs, not by starting from scratch.
Read the whole post for more detail on those points and some interesting history on Netscape.
6. Why you should (almost) never rewrite your software
This is another anti-rewrite opinion (you need only Google "refactor vs rewrite" yourself to see that a lot of people share this opinion). It puts its information about why people tend to want rewrites and why it's not a good idea in a succinct bulleted format.
One of the great things about refactoring is that you get to pick how much of your time you allocate to cleaning up existing stuff and how much you allocate to responding to new market demands. (And please spare me the excuses that you don't have the discipline to refactor, you're better than that). Should some massive change happen in your market (like a worthy new competitor), if you are refactoring a piece at a time, you have a fighting chance of responding. If you're in the middle of a rewrite, you're probably toast.
—Dharmesh Shah, Why You Should (Almost) Never Rewrite Your Software
7. Refactoring is an enabling activity
This blog post from 2004 called "Refactoring vs Re-architecting vs Redesign vs Rewriting" takes an interesting perspective, showing that refactoring is more about keeping things simple and flexible than it is about getting things right.
Refactoring does not involve changing the end result of the code, as mentioned previously. It's simply meant to make the code more readable and modifiable so that when you come back to make changes, there are fewer dependencies affected throughout the rest of the codebase and it's easier to understand what you're doing. Thus, refactoring is not about fixing bugs or adding features—it's an enabling activity.
—Robert Watkins, Refactoring vs Re-architecting vs Redesign vs Rewriting
8. Simplicity is relative
The two-part blog series "What is simplicity in programming?" is important to read as well. It serves as a reminder that simplicity doesn't feel the same for everyone.
I actually feel physically uncomfortable about working with code that I’ve not read. So maybe that’s why I am happier with one larger method than several smaller ones sprinkled across various classes and files. (To be clear, I am not advocating big functions and classes — no-one wants to read 300 lines in a single method, or classes with 50 methods. I’m in favour of biggER functions and classes than Martin Fowler is, that’s all.) ...
Depth-first/bottom-up is all about understanding the details, and letting the high-level picture emerge from them; Breadth-first/top-down is about grasping the overall shape of the system, and letting the details look after themselves (or at least, coming back to them later).
...So since I like to learn computer architectures and MVC frameworks depth-first, and since I read novels depth-first and listen to music depth-first, I suppose it’s not surprising that I like to read code depth-first
—Mike Taylor, What is “simplicity” in programming?, redux
9. Refactoring and restructuring creates code smells
Reads 9 through 11 are centered on the same theme: Making major changes to your code, even if its in small increments, will sometimes create temporary code smells as a byproduct. These "code appendages" should be eliminated as the new intended design starts to come together.
A refactoring sometimes introduces a bunch of appendages into your code. Are there any no-longer-necessary parameters passed into refactored methods? Are there local variables that are now lying around with nothing specific to do? For instance, if you’re moving a piece of functionality out of a method that’s doing too many things, it’s very likely that the original method now has extra references that just aren’t needed anymore.
In the midst of the refactoring, you may be so focused on the extraction, that you forget to clean these bits up. And, by cleaning it up, you may uncover a trail of other code that can go into the trash bin as well. Without this cleanup, you’ve just added the debt of useless code to your system. That might make the newly refactored codebase just as hard to read.
—Ka Wai Cheung, The Unintended Byproducts Of Refactoring Code
10. Intermediate code
A recent essay shows how restructuring and refactoring are often conflated among developers. The "big refactoring" mentioned in this essay is an oxymoron when you look at the Fowler definition. To reduce the amount of smelly, intermediate code that the author finds psychologically burdensome, make sure refactorings are as small as possible.
Listen to this blogger too and avoid the "big refactor."
In any big refactoring, you have to cope with two dissonant ideas:trying to make a codebase better and the reality that—in the midst of it, the code may actually feel a whole lot worse.
Ka Wai Cheung, The psychological battle of big refactorings
11. Chaos after every change
This article puts the "chaos" of code changes into a very useful visual perspective. Even if changes are small, there's still a small window of added complexity and chaos where the code could encounter problems. But the graphic in this shows how small-step refactoring introduces less total chaos into the system than a complete rewrite or "big refactor."
—Image from Michael Dubakov's Refactoring vs. Rewrite
12. Refactoring as a task before new features
Michael Feathers also has some of the best advice for refactoring. To summarize, he says that refactoring as you go is how you can improve your system and add features at the same time. Writing and maintaining automated tests is the secret weapon here. It's a time investment, but the maintainability benefits outweigh that investment significantly when it's time to redesign your system.
We can introduce a "hand off" in development. Here's how it works. For every story/feature that you accept in planning imagine how you would implement it if the code was in a state where it would accept the feature easily. Then, create a task for that refactoring work and make sure it is done before the other tasks for the feature, and (here's the trick) it should not be done by the whomever is going to do a feature addition task on the same story.
Here's the rationale. Breaking out refactoring at the task level highlights it. It's easier to focus on refactoring as as team when it is seen as separate activity.
—Michael Feathers, Refactoring is Sloppy
13. Types of refactorings
What are all the code smells you need to watch out for? What are all the refactoring techniques you need to know? This page has a huge reference list that answers both of these questions with helpful illustrations. The site offers online video courses to further explore refactoring, but the text tutorials are all free.
14. A checklist on when to refactor
With just a few simple questions and a scoring system, this worksheet can help you decide if something is worth refactoring.
15. Refactoring tips in an example video
This two-hour video shows you exactly what good refactoring looks like.
16. GitHub's interface refactoring
Here's an interesting, in-depth look at how GitHub employed refactoring practices from a UI/UX perspective.
That’s the beauty of refactoring an interface: after you clean up all of your informational and visual debt, you have so much more room to imagine new futures for your product.
—Ian Storm Taylor, Refactoring Github’s Design
17. Refactoring to microservices
The major trend in refactoring today is around developers who want to refactor their monolithic architectures into loosely coupled microservices. There's plenty of early theory (from Fowler) and case studies (this link is to a SoundCloud series) on this topic. Some other excellent reads are James Ward's Refactoring to Microservices and Camille Fournier's very recent Microservices: Real Architectural Patterns: A dissection of our favorite folk architecture.
Code Simplicity author Max Kanat-Alexander told TechBeacon about whether software industry's new focus on microservices changes the conversation around refactoring:
"One thing that I've advocated sometimes is rewriting a monolith incrementally by factoring out pieces of it in a series to become microservices. It's more or less the same as any other refactoring with the added concept that you're putting the function call across a process boundary or through some form of RPC. There are also paradigms that reduce those costs."
Asked if having a microservices architecture made complete rewrites of individual, small microservices more common and less problematic, since single microservices could more easily pass the five requirements for rewrites outlined in his book, he said:
"I think what you've said about how refactoring and rewriting apply to microservices is essentially true, except that I usually hope that rewrites aren't necessary at all for a system. They've never been necessary on any system that I've been in control of from the start. Even a 'micro' service can become pretty large. In a system that totals millions of lines of code, the 'micro' service could end up being 50K lines. I would hope that most of them are smaller, but I can imagine that some wouldn't be."
Kanat-Alexander also brought up some interesting refactoring challenges you might experience when you already have a microservices architecture:
"One of the key points becomes figuring out who is actually calling your service with what combinations of parameters. It turns out that you kind of need to know the identity of every caller—something that was usually pretty easy to do in a monolith but can be very easy to overlook in terms of logging in a microservices system."
Anything you'd like to add to the refactoring vs. rewrite conversation? Any essential reads you think should be in this list? Share them in the comments section.