In March, I participated in the ISE Southeast Executive Forum and Awards. The project that won the Southeast Project of the Year award was Cox Automotive’s Rugged DevOps program, which has integrated application security testing (AST) with development and delivery processes and automated the security validation process with its application security testing suite. This approach enables the company to enhance the security while its development teams are shifting to lean concepts such as agile, DevOps, and continuous integration.
What caught my attention, however, was its positioning of security defects and vulnerabilities as code quality defects and its partnership with the quality assurance (QA) team. I’ve had doubts about the effectiveness of this approach but this company's success in framing the program this way has changed my mind. Here's why.
What’s the difference between quality and software security assurance?
My experience has been that quality assurance teams struggle with supporting AST activities because security tests are different from functional and performance tests. In functional and performance testing, the expected results for test cases are documented before testing begins, and the quality assurance team looks at how well the expected results match the actual results. Security cares about vulnerabilities or weaknesses in the software that could lead to misuse or exploit. Before going further, let’s get grounded with some definitions.
Quality | |
Bug | Defect in a system or a representation of a system that if executed/activated could potentially result in an error (ISO/IEC 15026-1:2013). |
Software Defect | A condition in a software product which does not meet a software requirement (as stated in the requirement specifications) or end-user expectations (which may not be specified but are reasonable). In other words, a defect is an error in coding or logic that causes a program to malfunction or to produce incorrect/unexpected results. |
Software Fault | An abnormal condition or defect at the component, equipment, or sub-system level which may lead to a failure (ISO 10303-226). |
Security | |
Software Vulnerability | A mistake in software that can be directly used by a hacker to gain access to a system or network (CVE). |
Software Weakness | Flaws, faults, bugs, vulnerabilities, and other errors in software implementation, code, design, or architecture that if left unaddressed could result in systems and networks being vulnerable to attack (CWE). |
Quality assurance (QA) testing is focused on whether the application is performing the functions that it is supposed to do—does it meet its requirements? On the other hand, software security is making sure that security is considered in every phase of software development to harden the application. AST is focused on identifying software weaknesses that could lead to exploitable vulnerabilities such as SQL injection, cross-site scripting, cross-site request forgery, buffer overflows, use of hard-coded passwords, weak encryption, sensitive data, and so on.
Security as a QA issue
Security-related defects in any form should also be viewed as a QA issue. One can make an argument that software with quality defects and faults is more likely to have security vulnerabilities as well. Poor code quality leads to unpredictable behavior. From a user's perspective, that often manifests itself as poor usability. For an attacker, it provides an opportunity to stress the system in unexpected ways.
A 2014 study by the Software Engineering Institute found that development groups with a strong focus on quality tended to have fewer vulnerabilities in their source code. Quality and software security are not separate worlds, but rather two sides of the same coin—the bug that manifests as a system failure today could be a vulnerability exploited by an attacker tomorrow. Software security is just another important part of building good software.
As observed by Forrester’s Amy DeMartine:
“Here’s the truth of it: Your customer-facing applications are being probed for weaknesses. Constantly. And they will continue to be probed as you introduce new features and functionality. Worse yet, malicious attackers are highly skilled, resourceful, and determined. And more often than not, we are leaving our applications open to attack.”
Software security and QA are both concerned with removing risks. Software security teams work to remove security risks, and quality assurance teams work to remove risks to quality.
Getting the dev team some attention from management
Development teams are measured by how well they meet their requirements within budget and schedule constraints. The challenge software security teams run into at times is convincing development teams that AST findings need to be addressed, since doing so can negatively impact a project’s schedule and budget. Security loses because development often does not understand security or control security's efforts. The software security team needs to educate in the context of development's perspective.
That’s what impressed me with the Cox Automotive security team's approach. Developer MBOs were aligned to producing high-quality, defect-free or bug-free application code. The security team got buy-in that AST findings were just another form of software defect that needed to be tracked, prioritized appropriately, and remediated. Application security is now part of the quality program at Cox Automotive!
"Recognition from both executive leadership and development leads that secure code is quality code and vice versa was critical in establishing the intrinsic value of application security testing,” says John Sewall, Senior Manager of Cox Automotive.
The security and QA teams worked together, and as a result, are now exponentially more powerful. This makes me wonder why I haven’t pushed harder for a merger like this with programs I’ve been involved with.
As Xerox's Kenneth Silsbee puts it:
“Not treating security vulnerabilities as defects is actually avoiding the realities of software development and Information security's still-developing role. The purpose of development is to make software that satisfies customer needs, and fixing bugs is part of it.”
This approach requires being an active participant in working the security defects and keeping focused on their priority, just like the development teams work their other defects. The result is a more security-oriented QA department and a more quality-oriented software security department, which will help remove risk and provide improved continuity.
QA and security: Better together
QA and security teams define nonfunctional requirements that developers need to adhere to. These nonfunctional requirements are the underpinnings for building security-minded development teams, and when QA teams work hand in hand with the security team at the outset, it can be quite powerful. Automated security testing should be seen as a core component in this process, as development management recognizes that quality defects are entry points to vulnerabilities.
The software security team can help the QA team conduct AST using static and dynamic testing tools (SAST and DAST)—or, as in the case with Cox Automotive, leverage an AST managed service to conduct the security testing. SAST, DAST, and interactive (IAST) security testing methods have advantages and disadvantages, which is why multiple methods are often applied to applications. Gartner analyzes security vendors’ AST capabilities for each method on an annual basis (see "Highlights of the 2015 Magic Quadrant for application security testing").
Faced with having to maintain software quality and security while accelerating innovation, companies like Cox Automotive are looking for new ways to further reduce overall application development program risk. Nobody said that software security would be easy, but treating software vulnerabilities the way development treats quality defects is a good start.
Keep learning
Take a deep dive into the state of quality with TechBeacon's Guide. Plus: Download the free World Quality Report 2022-23.
Put performance engineering into practice with these top 10 performance engineering techniques that work.
Find to tools you need with TechBeacon's Buyer's Guide for Selecting Software Test Automation Tools.
Discover best practices for reducing software defects with TechBeacon's Guide.
- Take your testing career to the next level. TechBeacon's Careers Topic Center provides expert advice to prepare you for your next move.
