Modern IT environments—increasingly distributed, heterogeneous, and dynamic—show no signs of becoming easier to monitor and manage. In fact, the opposite is true, as organizations deepen their use of cloud computing, containers, and microservices. Faced with mounting IT complexity and legacy monitoring tools that fall short, IT leaders are turning toward AIOps — the application of artificial intelligence to IT operations.
Your IT peers are buying into AIOps tools in a big way: MarketsandMarkets Research estimates that the global AIOps market will grow from $2.55 billion in 2018 to $11.02 billion by 2023, a jump representing a compound annual growth rate of 34%. And according to a May 2020 study from Digital Enterprise Journal (DEJ), there’s been an 83% increase since 2018 in the number of organizations deploying or looking to deploy AIOps.
"The growing interest in AIOps is a positive sign for the market's future."
—Dennis Drogseth, VP of Research at EMA
The promise of AIOps is compelling: a radical improvement in the speed and precision with which IT problems are detected, diagnosed, and resolved, resulting in dramatically fewer and shorter outages of critical apps and digital services. But successfully adopting and deploying AIOps is far from straightforward.
Here are six key trends that IT decision makers should watch as they plan and refine their AIOps strategies in 2021.
1. Expect more AIOps hype—and confusion
As often happens with technology terms that gain marketing buzz, AIOps can be defined in different and often self-serving ways. In the DEJ study, 64% of respondents found the AIOps solutions landscape "confusing." EMA's Drogseth agrees that the term has been controversial and unclear.
Experts say this will only worsen in 2021, as AIOps' popularity grows and more vendors claim to offer it in some shape or form. That means it will be more difficult for IT leaders to cut through the cacophony of carnival barkers hawking their AIOps wares and get started with AIOps.
You'll need to be particularly rigorous and diligent in your research and assessments when choosing the right AIOps tool for your organization, said Micro Focus product manager Gary Brandt.
"The challenge for customers is trying to discern the real value, the ROI of AIOps, versus the hype."
—Gary Brandt
Expect next year to bring with it a dose of reality, said Lars Rossen, CTO of Micro Focus—and, inevitably, a sense of disillusionment.
"People think AIOps will magically solve all their problems, but that's not going to happen."
—Lars Rossen
The right way to think about AIOps is as a continuation and extension of the IT operations analytics technology that has been built up over the past 20 years, Brandt said.
"Whether that's monitoring or event management or problem isolation or automation, it's not changing the 'what' you're doing, but the 'how.' It's really enabling you to do it better."
—Gary Brandt
IT leaders should also know their reasons and goals for adopting AIOps. "What are you trying to improve in your hybrid IT environment? Be very clear about your objectives," Brandt said.
Torrey Jones, principal consultant at the Greenlight Group, said that it's important for organizations evaluating and implementing AIOps to clarify with their vendors what AIOps is to that specific vendor.
EMA's Drogseth prefers the term "advanced IT analytics" to AIOps because it indicates that the scope goes beyond IT operations, and includes DevOps, IT service management, security operations, and business stakeholders. He also sees the AIOps space less as a traditionally defined market and more of a "landscape" whose vendors represent a broad variety of approaches, so that determining which one is the best choice will vary widely depending on the organization.
EMA outlined a useful set of core criteria for AIOps products and services in its recently published report "AIOps: A Guide for Investing in Innovation," including:
- Assimilation of high volumes of data from cross-domain sources
- Access to critical data types, such as events, logs, and configuration data
- Self-learning capabilities to deliver predictive, prescriptive, and if/then actionable insights
- Support for a wide range of advanced heuristics
- Usability as an overlay to consolidate multiple monitoring tools
- Support for private and public cloud, as well as hybrid/legacy environments
- Support for multiple use cases
2. AIOps will prove useful for security
The potential for AIOps to extend its reach into the cybersecurity realm has been on the horizon for years, but 2021 will be the year that potential becomes reality. By bridging the gap between IT operations and security operations, AIOps will further boost system uptime and reliability, said David Linthicum, chief cloud strategy officer at Deloitte Consulting.
"Considering that operations and security should work together, there is no greater advantage than to have an operations tool that can talk to the security manager and the other way around."
—David Linthicum
For example, by having visibility into security data, AIOps could detect that, say, an application performance issue is being caused not by an IT glitch but by a cyber attack against the underlying server, because a malicious hacker exploited an unpatched vulnerability. "That particular situation should kick off security processes to mount a defense. For traditional tools, these would play out as performance issues, and not make the link with security threats," Linthicum said.
This will provide the advantage of leveraging operations as a first line of defense, such as shutting down a server that's under attack or shutting off access to a storage system that's compromised, Linthicum added.
3. Tool vendors will consolidate
As the AIOps market gets more crowded, a shakeout is coming. One dynamic fueling consolidation could be that large providers of traditional monitoring tools want to add AIOps capabilities to their products if they haven't done so already, making the smaller, pure-play AIOps players an acquisition target. "That should result in better AIOps technology for customers," Linthicum said.
AIOps as a concept and as a technology category represents the maturation of ops monitoring tools. In recent years, most vendors in the traditional ops and CloudOps space "bolted an AI engine onto their tools and called it AIOps, whether they leverage AI systemically or not," Linthicum said.
Then there are the purpose-built AIOps tools from startups that leverage AI from inception. Which tool is best for you depends on your organization's needs.
The startup tools tend to be more innovative and to leverage AI more effectively, and they support more modern systems such as cloud platforms. Older tools that have been recast as AIOps tools tend to support legacy systems much better and are typically the choice for It operations folks who want to manage both traditional and cloud-based systems with a single tool.
"They are complementary, and many enterprises use both types to cover all operational bases."
—David Linthicum
4. DevOps pros will embrace AIOps
AIOps has been aimed primarily at IT operators, but its appeal has progressively widened to DevOps teams, which need advanced tools to monitor their complex environments and the raw, granular, and extensive observability data—logs, metrics, traces—that they generate.
By applying AI and machine-learning (ML) algorithms to your observability and monitoring data, AIOps becomes a part of the DevOps tool chain, focused on monitoring and managing testing, performance, security, and so on. Taking this approach also provides "real-time feedback using automation to both integrated tools and DevOps engineers directly," Linthicum said.
Nancy Gohring, a senior analyst at 451 Research, a part of S&P Global Market Intelligence, said that in the firm's recent survey of DevOps practitioners, 42% defined AIOps as a tool that analyzes logs, traces, and metrics using AI and ML.
"That starts to align with how people define observability, so there may be an interesting overlap between AIOps and observability in the DevOps crowd."
—Nancy Gohring
5. Automation capabilities will broaden
Automation has been a core capability of AIOps offerings from the get-go, but automation can take many different forms. While AIOps offerings aren't at the stage where human intervention has been fully eradicated, expect to see advances in this area in 2021.
These days, more products have automation that's tied to a problem-resolution type of use case or workflow. "Now we see how we can trigger that or drive that through analytics," Micro Focus' Brandt said. "So I see automation as a big trend going forward."
For his part, Micro Focus' Rossen highlights four automation areas where he expects to see progress both in the technology and its use in the enterprise:
- The "classical" type of AIOps automation, where logs are massively ingested and then analyzed with ML algorithms to detect anomalies against baselines
- Robotic process automation, where fixes are triggered without human intervention
- Analysis and correlation of topology data to see how systems are connected and then use that information to discover root causes of problems
- Automation to help end users intuitively fix problems through smart virtual agents in, for example, automated help desks, making customer support more precise and useful
Deloitte's Linthicum points to advances in self-healing capabilities. Some AIOps can "heal issues found with systems that are managed or monitored," he said. If the AIOps system finds an issue, a process is launched to attempt to correct the problem, such as restarting a server or a network hub.
"The trend is to move to active, or self-healing AIOps tools moving forward."
—David Linthicum
6. Use of collective intelligence will deepen
Another important trend to watch is what 451 Research's Gohring calls "collective intelligence," wherein an AIOps vendor analyzes in aggregate all the monitoring data from its customers to identify overall trends that it can share with everyone. "They analyze this data across their customer base, derive these benchmarks and offer insights," Gohring said.
For example, a vendor could see performance metrics from a customer trending in a particular direction and be able to predict, based on collective knowledge, that it will turn into a problem and so should be looked at. Or, she said, suppliers could analyze how customers repair a specific issue and share that finding, so other customers learn how other IT shops are addressing the problem.
AIOps will be trending 2021
As 2021 approaches, IT leaders grappling with a complex environment that's increasingly difficult to monitor should keep a close eye on AIOps—whether they're still in the planning stages or have already started deploying it.
It's an overhyped technology that may fall short of inflated promises from vendors, but has nonetheless proved effective in the real world at automating and streamlining IT operations.
Moreover, as the trends outlined here indicate, AIOps technology is evolving and improving, its scope is broadening to include areas such as DevOps and SecOps, and its use cases are growing.
Ultimately, AIOps is aimed at helping IT teams with some of their most important missions and most pressing priorities: ensuring the reliability, stability, and uptime of the applications and digital services that have become critical to business success.
Keep learning
Choose the right ESM tool for your needs. Get up to speed with the our Buyer's Guide to Enterprise Service Management Tools
What will the next generation of enterprise service management tools look like? TechBeacon's Guide to Optimizing Enterprise Service Management offers the insights.
Discover more about IT Operations Monitoring with TechBeacon's Guide.
What's the best way to get your robotic process automation project off the ground? Find out how to choose the right tools—and the right project.
Ready to advance up the IT career ladder? TechBeacon's Careers Topic Center provides expert advice you need to prepare for your next move.
