The velocity of innovation afforded by DevOps is one of the most fundamental drivers of IT's shift to this new mode of software development and delivery. But if an organization forgets to effectively account for quality in a pell-mell dash to DevOps, it's missing half the point. DevOps practices are intended not only to improve the cadence of feature releases but also to reduce defects. In the end it isn't just about continuous delivery, but overall continuous improvement of software as well.
However, with DevOps' emphasis on an automated pipeline, it's clear that the traditional means of improving quality are being called into question just as surely as is the old relationship between devs and operations staff. Experts say that separate QA organizations with teams of testers performing their duties during discrete testing windows are quickly becoming antiquated.
And this scares the heck out of QA professionals.
The truth is that there is still plenty of room for testing professionals in the era of DevOps. They just need to adapt to a rewritten organizational hierarchy—and a completely different mindset about testing objectives.
"It's a tough message to hear," says Ernest Mueller, lean systems manager for AlienVault and organizer of DevOpsDays Austin. "It's a lot of the same kind of fear and uncertainty that operations people faced with DevOps: 'Well, is this putting me out of job?' The answer is, 'Not necessarily.' But it will if you insist on just doing what you have always done."
QA pros will still champion quality
When implementing DevOps, many leading organizations are leveraging new practices and automation to increase both speed and quality. For example, at the DevOps Enterprise Summit in October, Tapabrata Pal, product manager for CapitalOne, noted that his organization is not only committing code more quickly, but it is also reducing defects, by approximately 60 percent since it started its DevOps transformation four years ago.
For many organizations, DevOps' secret ingredients are a test-driven development approach and continuous testing. Even when organizations don't start DevOps with an eye toward improved automated testing, they quickly learn through QA and integration bottlenecks that this is step one toward achieving continuous delivery of code. In most cases, developers are more responsible for the care and feeding of their tests.
"From day one, developers are writing their own tests; they don't pass Go until the function, or whatever it might be, is without errors," says Andrew Storms, vice president of security services at New Context, a consultancy focused on helping companies develop more rugged software. "That in and of itself already covers a ton of the testing mechanism that's been historically done by QA." Which is why QA professionals are pausing, wondering where this leaves their role in the new scheme of things.
The buggy whip analogy
According to Mueller, the smart testing professionals shouldn't break out into cold sweats prematurely. There's still going to be plenty of need for those with a QA-intensive focus and extensive testing experience. But these people likely won't be working in the same role anymore. He calls this a classic "buggy whip problem," referring to the transition more than a century ago from horse-and-buggy transportation to automobiles.
"I know you like making buggy whips, but that's not going to be a job in the future," he says. "You don't have to become a professional developer for a modern DevOps role. But you do have to change your mindset and start helping people instead of providing a transactional service to them. Luckily, the vast majority of people in QA are capable of making that pivot."
So instead of running the tests and handling the hands-on work, QA pros will transition into a more consultative and distributed role to help developers learn how to write better tests and improve their approach to screening for quality. Because, at the end of the day, developers are still developers. They've not been trained to look for quality issues. And even though they care about it, they're going to miss things.
"In QA there's plenty of opportunity for expert consulting on things that Joe Developer doesn't necessarily know," Mueller says. "If you ask a developer to do, say, a performance test, you're likely to get something that a test professional would look at and say, 'That was random and half-assed.'"
The move to mentoring
Avoiding those half-baked tests will require education and guidance. Mueller has the perfect example of the value that QA adds to a DevOps environment from his current work life. He just started a new job at AlienVault, and one of the first action items he needed to work on had to do with developer-written tests. At the moment, his team is working on a new front end written in AngularJS. And writing Selenium tests for this environment has proved to produce very fragile tests. So now he's got QA cooperating with developers to find ways to bolster test resiliency.
"So we're working with the developers to say, 'Hey, each UI element, you really need to put an identifier on that and not have us rely, for example, on the XPath that's going to change, because that will make tests more robust,'" he says. "And they're like, 'Oh yeah! I hadn't thought of that.'"
QA gets creative, more strategic
In order to account for a more distributed model of testing expertise and responsibilities in DevOps mode, QA leaders should look to position themselves organizationally so their work can add the most value possible in the software development life cycle. This means obtaining new and meaningful responsibility for making higher-level strategic decisions regarding testing and quality issues. And yes, that could mean a change in the org chart.
"We're seeing QA move earlier into the story acceptance criteria. The story doesn't get put into the sprint unless it has acceptance criteria that is actually being reviewed by someone on the quality team, and being reviewed by someone in security, and someone in performance and reliability."
From a logistical perspective, Mueller says, there are three likely ways to see QA roles reassigned to better fit DevOps organizations.
- First, there's the opportunity to embed QA expertise directly within product teams to directly support them. This is an area that has been most effective in dealing with QA transition so far, says Steve Anderson, CEO of the DevOps and agile consultancy Clutch."The most successful ways I've seen where you still get a hybrid going on is where a QA is directly embedded into the sprint team or the scrum team, where they're involved from the get-go," he says. "They get the user's story. They're writing test cases."
- Second, Mueller says, there's also the potential for diverting QA pros into tooling groups responsible for writing the test harness or providing tooling across the organization.
- And third,there's the opportunity for creating a center of excellence or consulting group meant to advise specific work groups on strategic issues of quality. "That can be tricky, because having the separate consulting group can easily turn into doing it the way you used to work," he explains. "It's a very subtle approach change that you have to make. It's not like they're not doing work for you. They're providing expert consulting so you can do the work better."
Changing the mindset
Regardless of how they think the org chart will shake out in their current jobs, QA professionals need to universally start thinking about how they can hone their skills to add value as more of their day-to-day work is automated. This means being ready to help elevate a test approach rather than executing it, Storms says.
"You're thinking about what the end user is doing, how can we mimic that, and what kind of data can we capture around that," he says. "It's not pass or fail; it's also instrumentation at each step throughout the SDLC."
The speed of automation should have QA working within a totally different paradigm, says Wayne Ariola, chief strategy officer for Parasoft, a testing and service virtualization firm. "Today we're mostly in causal observation mode: Build the test, experience the defect, collect the defect—it's causal. Something happened, we collected it," he says. "We need to move to a more probabilistic mode."
This means that more operational data needs to move down into the organization for it to have a better view of expectations. For example, whereas in the past QA was focused strictly on defect documentation, now that focus should be on taking data from a broader scale of automated tests and using it to expose the most likely causes for those defects and where they're likely to crop up again if the causes aren't addressed.
"This means getting to know the internal application much more as testers than we do today," he says. "This also means we have to understand the dependencies associated with the operational testing."
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
