In the agile era, an untidy assemblage of commercial software suites, open source apps, and custom-built in-house tools has become the norm for most organizations. A recent survey by DevOps.com and Automic found 69 percent of respondents said their tool chain is steadily growing, and 82 percent said many of those are outside of the officially supported tool stack.
As those numbers reveal, "tool sprawl" is a significant issue, and odds are your organization isn't immune. "It’s getting pretty widespread," said Kelly Emo, director of application testing solutions, product and technical marketing at Micro Focus. "Just about any large organization that has faced a significant digital disruption in the last few years is going to run into it."
What's behind this troubling trend? We talked to the experts to explain how we got tangled up in tool sprawl and what we can do to rein it in.
The temptation of more tools
In one sense, tool accumulation is a fact of agile development life. Teams use multiple tools out of necessity, requiring specialized solutions for their particular domain. It's common for even small companies to have five to 10 different tools in use, while more than half of large enterprises use a whopping 20 or more, according to the Automic survey. And with new options becoming available all the time, teams find it easy to add to the toolchain as a project progresses. Many of these tools are used to solve a particular problem once and are never used again.
"Every day dozens of new applications make their way to the market and engineers are tempted to try them all," said Dmytro Sychevsky, a DevOps team lead at PDFfiller. "Each of them aims at a specific problem: You use Pinba when searching for bottlenecks, New Relic to check out how your app works in the user's browser, Kibana plus ELK Stack to analyze the issues in your product, or Zabbix for system monitoring."
"Eventually, you end up with a massive pile-up of apps, half of which were only used once and now rest in peace—holding up precious storage capacity.”
—Dmytro Sychevsky
Just as often, it's fealty to a technology rather than fickleness that drives the accumulation of tools, said Nilesh Patel, manager of testing services at KMS Technology.
"People feel more comfortable and effective when they use tools they have experience with," he said. "They will request to work with those tools just because they're comfortable and don't want to learn a new tool." When this happens you end up with multiple tools used by multiple people in the department to do the same things.
Mergers and acquisitions also contribute to the problem. Emo cites a common scenario where disruption drives a company to take on new developers and technologies, inheriting new tools along with them.
"Business teams have to respond," Emo said. "'I've got to get a mobile app because my competitor has a mobile app out, and if I don't have a better one, my customers are going to go to my competitor.' And so they're going to boutique development firms, they're hiring out of college, they're acquiring small Silicon Valley ISVs or ISVs out of Eastern Europe—and the tools come along for the ride."
More tools, more problems
Using a menagerie of tools can introduce a host of problems into a testing environment, from functional overlap to integration difficulties to reporting inefficiencies to hidden-costs associated with tools that get abandoned after limited use—to name just a few. And there's the simple fact that the more tools you have, the more complex it is to manage them.
Using more tools means you have to learn how to use each of them. To compound this issue, it creates a level of "tribal knowledge" as expertise accumulates in just a few people across various teams, said Rob Szumski, product manager at CoreOS.
"Sprawl incurs a large technical debt around test orchestration."
—Rob Szumski
If those knowledge-holders leave, it creates a huge issue when it comes to bringing a replacement up to speed. Yes, there will be artifacts and scripts left behind, but the guiding principles for why are easily lost. "These skills and knowledge aren't always transferrable between teams, as they may touch on different levels of your application's stack," he added.
Dialexa vice president and former QA lead Luke Gordon warned of the ultimate impact this can have on the organization as a whole.
"Tool sprawl is a nightmare for management."
—Luke Gordon
That's because you have to deal with licenses, educate team members on how to use each tool, define processes around them, and so on, he said. Each tool a company introduces to its repertoire decreases the net business value a company can create and places the focus on the approach rather than the problems the company is trying to solve.
One of the most potentially damaging drawbacks of tool sprawl is the lack of visibility and quality control it can impose, particularly when lines of business are increasingly developing their own projects independently from central IT.
At some of these companies, the central IT group is using a center of excellence product, and the disparate line of business teams are using open source or popular commercial tools for development teams like, but "nobody’s connecting them all together," she said.
"What happens over time is the financial aspect comes into play and the CFO says, 'Why are we spending all this money on these disparate tools?'" she added. Maybe they have a situation where a line of business developed part of an application that they now want to integrate with a different application to expand their business value and they can't get them to integrate. Or they run into a quality issue because they don't have end-to-end visibility.
And the questions arise: "'Why aren't we using tools that can integrate together and give me a common view and governance?' It gets really expensive with tool sprawl.”
Taming the tool box
If you’re tackling tool sprawl, resist taking an authoritarian approach, Emo said.
"You’re not going to be successful as a QA leader going into a business unit saying, 'I am dictating that you have to use one of these three tools and we need to rip out your installation of Atlassian and Selenium.'"
—Kelly Emo
Rather, she suggested, QA leaders should start with a more holistic approach.
First develop a culture of quality, which is easier said than done, she said. "It's engaging in a collaborative way with the business leaders to enable them to see the benefit of a more disciplined approach to quality. It's getting them thinking, 'How do we engineer for quality in our processes' rather than 'We'll just do lean agile, we'll push things out fast, and if we run into a problem we can roll it back.'"
The experts agree that on a tactical level QA leaders need to start by getting a handle on exactly what tools are being used across the organization and evaluating them in terms of cost, compatibility, ROI, and how much they're used. Prioritize the top tools, retire the obsolete and little-used ones, and look for ways to automate and standardize processes.
This should be a collaborative undertaking, said Gordon.
"This needs to be a team focus rather than just a specific group. By making this a collective effort, teams introduce a sense of accountability and ownership in the direction they move forward."
—Luke Gordon
A sense of accountability on its own can go a long way toward curbing tool sprawl. Making sure that all tools are owned and maintained by the developer teams that introduced them is a good start, said Truss co-founder and CTO Mark Ferlatte.
"When teams have to trade between working on features and doing tool maintenance, the urge to introduce new tools that are, at best, marginally better than existing ones tends to fade."
—Mark Ferlatte
Controlling tool sprawl can be challenging, but the benefits—reduced costs, improved efficiency, and an increase in the value departments deliver—are well worth the trouble.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
