When New York City's Cyber Command (NYC3) introduced threat modeling into its process, the organization saw significant improvements. In the first four months it blocked an additional 541 intrusion attempts, caught hijacking attempts of five privileged user accounts, and found three server vulnerabilities.
NYC3's presented all of the details recently at the USENIX Security conference.
Working with researchers from the University of Maryland at College Park and Wake Forest University, NYC3 taught a group of 25 security engineers, analysts, support staff, and IT leaders how to apply threat modeling to their daily responsibilities. After 30 days, 23 participants still used the concepts in their daily work.
Before the training, the security professionals were so busy responding to current threats that they were unable to codify exactly what their plan of action should be, said Rock Stevens, a computer science PhD student at the University of Maryland (UMD) and primary author of the USENIX paper.
"We were surprised at how many new things the participants came up with by just spending an average of 37 minutes analyzing each threat."
—Rock Stevens
Here's why threat modeling matters, and four lessons from NYC3s experience.
Do try this at home
Security experts have long recommended that security teams incorporate threat modeling into their process of analyzing their corporate assets. Taking a structured approach to analyzing risks to information infrastructure can reveal new threats that aren't contained in compliance requirements or industry standards.
Jeremy Batterman, global leader for the threat intelligence Fusion team at Trustwave SpiderLabs, said a big part of the underlying purpose of threat modeling from an enterprise perspective was visibility.
"[The] more visibility, the more knowledge you collect, the better you can react to threats."
—Jeremy Batterman
The NYC3 protects an organization the size of many large enterprises. It secures the City of New York's 143 separate departments, agencies, and offices and the more than 300,000 employees who support the city's 8.6 million residents.
As a result of the introduction to threat modeling, the NYC3 instituted more rigorous disaster recovery exercises, implemented two-factor authentication on previously unprotected subdomains, instituted a bug bounty for its public-facing web server, and added sensors to 1,331 unmonitored endpoints.
Other organizations have used threat modeling to gain similar insight into what assets need protection.
We use threat modeling as a way to direct our efforts, said Michael Feiertag, CEO and co-founder the web application security company tCell.
"Threat modeling starts as a very theoretical exercise to focus on what you need to prioritize, but highlights actionable recommendations."
—Michael Feiertag
Four lessons from the case study
1. Don't get caught up in framework selection
The first step was an initial questionnaire to establish the participants' baseline attitudes toward threat modeling and how efficient they believed themselves to be at their jobs. Then the instructors spent one hour teaching small groups about practical threat modeling.
The researchers selected the Center of Gravity threat modeling framework, which starts by identifying an overall object, or end state, for the organization or area of responsibility.
Then the analyst outlines the assets, services, and tools needed to accomplish the objectives, selects the most critical asset and designates that as the center of gravity (CoG). All other assets are classified as either critical capabilities (services provided by the CoG) or critical requirements (assets that the CoG needs).
Finally, the practitioner identifies the critical vulnerabilities that could cause each asset to fail and builds an actionable defense plan (ADP).
Other effective threat modeling frameworks exist, but the researchers did not have a large enough study group to compare two models in the enterprise. Companies should pick a threat model with which they have practical experience, said UMD's Stevens.
Frameworks can enable organizations to prioritize where they spend their time on real threats, he said. "It is not necessarily giving them back hours in the day, but they are getting more time to spend on developing a more proactive security posture."
2. Hands-on training helps learning retention
Participants were taught the CoG framework and how to apply it as a threat modeling technique. The instructor showed each class an example—determining the CoG for Star Wars' Galactic Empire—to illustrate the thought process. Then, smaller groups worked with instructor guidance on creating a CoG model of a fictional e-commerce company.
Following the class, participants spent 60 minutes applying CoG to their daily duties, to provide hands-on reinforcement of the lessons. About half of the participants said threat modeling allowed them to better understand critical assets and capabilities, as well as new vulnerabilities in their work sphere.
Of the rest, seven participants said they did not find out anything new but found that threat modeling helped reinforce defensive concepts, while four participants said it helped emphasize specific issues.
Several participants reported that threat modeling gave them the tools to better monitor critical assets, identify threats, mitigate threats, and respond to incidents.
Michelle Mazurek, an assistant professor of computer science at UMD and the advising author on the paper, said it was effective with very little training.
"We gave people a couple of hours to think about security in a structured way, so it would not be surprising if we did not get good results. It was surprising that little training produced such significant impact."
—Michelle Mazurek
3. Threat modeling brings real benefits
The researchers analyzed data from NYC3 logs after 120 days to determine what impact threat modeling had on the organization's security. A crowd-sourced bug bounty program for a public web server—suggested as a result of the threat modeling exercises—found three previously unknown vulnerabilities, for example.
By deploying 1,331 sensors to previously unmonitored endpoints, the security team caught 541 unique intrusion attempts, 59 of which were considered critical events. All 541 attacks could have been successful if sensors had not been deployed as a result of threat modeling.
Finally, the exercises also identified a domain that did not have two-factor authentication enabled. During the 120-day period, they detected seven attacks against privileged accounts in that domain. Of those, five failed because of the new two-factor requirement. "Some of the benefits that we saw were around communication, in terms of participants opening discussions with other people on the teams and bringing issues to higher-ups," said Mazurek.
4. Modeling identifies non-technical threats, too
Following the adoption of the CoG framework, participants identified several broad threat areas, including many that weren't technical. Two participants, for example, identified untested disaster recovery plans as potential vulnerabilities, inspiring NYC3 to comprehensively test fail-over systems in the case of natural disaster or terrorist attacks.
Another significant issue is the potential to lose institutional knowledge when an employee leaves the NYC3. Reliance on legacy systems and unmonitored privileged accounts could both exacerbate the risk when workers change jobs.
Some of the technicians recognized as a weakness that those relationships—which you need to be in place when you have an incident—"might go with the people, if they moved," said UMD's Stevens.
"As a consultant, in the back of your mind, you understand that this is part of the organizational culture, but to directly tie that to a security risk is something else."
—Rock Stevens
Prioritizing security issues is just good business
While the researchers were quick to emphasize that their research is only a single case study, the results should be applicable to companies of all sizes.
"The goal of prioritizing your resources and being efficient seems to be good for just about any company. It seems to me that focusing on the big picture and prioritizing is a good lesson for any company."
—Michelle Mazurek
Keep learning
The future is security as code. Find out how DevSecOps gets you there with TechBeacon's Guide. Plus: See the SANS DevSecOps survey report for key insights for practitioners.
Get up to speed fast on the state of app sec testing with TechBeacon's Guide. Plus: Get Gartner's 2021 Magic Quadrant for AST.
Get a handle on the app sec tools landscape with TechBeacon's Guide to Application Security Tools 2021.
Download the free The Forrester Wave for Static Application Security Testing. Plus: Learn how a SAST-DAST combo can boost your security in this Webinar.
Understand the five reasons why API security needs access management.
Learn how to build an app sec strategy for the next decade, and spend a day in the life of an application security developer.
Build a modern app sec foundation with TechBeacon's Guide.
