Threat Modeling Knowledge Bases and Templates

24 Apr 2022

Threat Modeling Knowledge Bases and Templates

A mix of frustrations, aspirations, and unsolicited opinions regarding the current state of Threat Modeling are initially what prompted me to write this post.

First some clarification up front. When I talk about threat model templates, threat knowledge bases, or custom threat lists then I’m roughly talking about the same thing. Threat knowledge bases are a database of pre-defined threats that capture the current threat landscape. The more precise a knowledge base aligns with your model’s use case, then the higher value it provides. Templates usually include a knowledge base while also including other things like the DFD elements, their properties, and logic to tie knowledge base with the DFD elements.

Pain Points

Threat Modeling, which is arguably the most fundamental and important Software Security Development Lifecycle (SSDL) activity, is far from a matured process. First, there is a lack of any useful + free tooling that facilitates the process entirely end-to-end. Next, all modeling practitioners have different methods, which are implemented with segregated and specialized knowledge. This makes facilitating the process end-to-end extremely difficult. Lastly, the execution of the activity can take a large amount of time and effort for the multiple parties engaged. This does not make sense from a cost/benefit perspective, especially if your team is attempting threat modeling for the first time!

Threat Modeling may seem ancient, decrepit, or somehow still better on a whiteboard. Due to this, it is continually being required by more and more guidance, as well as the fact that more regulatory agencies are demanding to see a threat model.

Some issues Threat Modeling faces are as such:

• How do we make Threat Modeling highly repeatable?
• How can an enterprise produce Threat Models at scale?
• How can the modeler deal with a constantly changing threat landscape?
• How can the completion of Threat Modeling be less reliant on hiring a 3rd party consultant who is in tune with that threat landscape? (not just for specialists)
• How can the newly hired security professional effectively produce a Threat Model of value?
• How can a small startup, with no dedicated security personnel, quickly produce a Threat Model of value?

I believe the answer to all these questions lies in threat modeling templates.

Let me explain…

What Works For Me May Not Work for You

Before I begin, I must say this caveat: what is presented in this blog reflects my particular views and opinions of Threat Modeling. These views should not be taken as prescriptive advice on “how to Threat Model’’, nor do these views reflect that of my employer. This is more so a reflection of opinion on the matter.

“All models are wrong, some models are useful” - George E. P. Box, British statistician

I personally find threat modeling with data flow diagram (DFD) templates to be extremely useful and if you find value in them too, then that’s great! Hopefully you find this post informative.

The Problem

At a high level, the problems with Threat Modeling can be distilled down to lacking 3 basic concepts:

1. Repeatability
2. Consistency
3. Simplicity

All three, when combined, can begin to achieve scalability. So, let’s dive into each.

Repeatability

Repeatability is desired in order to produce the same high quality product over time. To make Threat Modeling highly replicable, we are able to create a model from a template.

` `A Threat Model DFD template contains the following:

• A library of non-generic DFD elements (called stencils in MS TMT) and their custom properties
  • Generic Threat Modeling elements are: datastores, flows, processes, external interactors, and boundaries
• A non-generic library of threats and their custom properties
  • Generic Threat Modeling threats can that of STRIDE or another methodology
  • Threat properties usually includes threat logic
    • Ex: ‘Flow is HTTP’

“In a nutshell, Templates are stripped-down Threat Models”, which don’t contain any diagram level information. “You can consider them as highly-specialized knowledgebases, and you can combine them to create your specific context”. Well said Simone, creator of Threat Manager Studio.

Templates provide a repeatable way for anyone to begin to address what they are working on and what could go wrong. Simone has a few interesting blog posts on designing and maintaining Microsoft threat model templates (see starting, stencils, threats, properties). But these were written 5 years ago when Microsoft released the 2016 version of the tool and sadly, templates didn’t catch on.

Consistency

Threat logic should be configurable and simple. If using a methodology, like STRIDE per-element, every threat’s logic condition is checked for every diagram’s element. If logic conditions for the threat are satisfied, then it qualifies as a generated threat, and should show up in the generated threat list.

Let’s run through a hypothetical, yet trival, example to put everything into perspective:

Say you have a template for AWS (Amazon Web Services). That template has a non-generic datastore element called “S3 bucket”. That S3 bucket element has a unique element property called “is public”. The threat logic can be something simple like: “S3bucket.is_public is ‘yes’”. Now when the modeler has the AWS template applied, uses the S3 bucket element, and then sets the “is public” property to yes, then the model will generate a non-generic information disclosure threat titled something like “An adversary gains access to data within a public S3 Bucket”.

Some have described the Threat Modeling process as:

• Identify Assets
• Create an Architecture Overview
• Decompose the Application
• Identify the Threats
• Document the Threats
• Rate the Threats

If using consistent architecture, and threat model templates that contain that architecture, the “Create Architecture Overview” step has been taken care of upfront. The Architecture is now all the non-generic elements within the template available for the model practitioner. Also, by building a template with a non-generic list of threats and using threat generation, the “Identifying Threats” phase of the process has been abstracted. This is because all relevant threats and logic are recorded within the template’s knowledgebase then generated within the model.

TM Tooling’s Role in Templates

Templates provide a map that enables the model practitioner to navigate a complex threat landscape

Threat Model tooling should never be responsible for producing templates themselves. Rather, this should be provided by the companies, working groups, and specialists, as the tool simply couldn’t cover every domain. Some tooling tries to provide a non-generic threat list, including threat generation, and this usually kills the tools overall usability. Only the specialist can extensively cover entire domains like mobile, embedded systems, cloud, OT, automotive, medical, etc.

The Microsoft Threat Modeling tool, and others, are plagued by the problem that no one contributes their template files back to the community. I’m not sure if there are more than 10 threat model templates currently available on the internet.. that’s sad.

Sharing threat model templates, and getting the community to contribute, is by far the biggest challenge of templates. Industry and domain experts should be the ones creating, investing in, and sharing these templates.

But who should model? Everyone!

Threat Modeling Manifesto

Which brings us to the Threat Modeling Manifesto.

Referencing the manifesto, templates first help assist with: “What are we working on?”. Not in terms of decomposing any single application, but in terms of describing a consistent set of system components, architecture, and environmental aspects.

Templates also address “What could go wrong?”, by using a threat list, threat logic, and threat generation to identify.

Simplicity

“Everything should be made as simple as possible, but no simpler” - Einstein quote that applies to modeling

Lastly, I also believe templates address an essential Threat Modeling Manifesto anti-pattern which is the Hero Threat modeler.

Given the speed of the security industry, the rapid change of the threat landscape, combined with the segregated and yet highly specialized knowledge required to continually identify “what could go wrong”, it’s becoming not practical for the threat modeler to traverse domains.

“A jack of all trades is a master of none.”

I see many teachers of Threat Modeling essentially imply that the hero threat modeler mentality is required to get the job done. That intimate knowledge of the entire threat landscape is needed. I mean who can blame them? They have a consultant business to sell and that’s what works for them. Getting value out of the process without a specialist can be impossible.

“A jack of all trades is a master of none, but oftentimes better than a master of one.” - The original saying. Which still applies here because on the other side, too many modelers that aren’t specialists and therefore are not accounting for relevant threats that fall outside their domain. Incorporating templates into threat modeling can potentially account for what the modeler didn’t account for and fill gaps for the unknown knowns.

“… that wasn’t in our threat model”

With templates, the specialists does some work upfront so that the modeler can do less. Also while enabling the modeler to be more versatile and adapt to their threat landscape. Rather than the modeler solely using their acquired knowledge, they can also be a generalist from a predefined set of templates.

Cons of Templates

The downside of producing a template is simply maintaining that template over time. As the threat landscape changes, a template could require capturing additional threats, additional diagram elements that begin appearing in new designs, or maybe just to modify an element property to contain more context about that element.

A downside of using template based models is being too reliant on the template’s threat list and threat logic. The threat list could be not as complete as the modeler thought. The threat logic could be flawed or have edge cases not addressed by the template designers. False positives can be identified as such within the generated threat list, but false negatives can be potentially disastrous.

Future Of Template Diagraming

I believe the future process of Threat Modeling with templates to rely upon:

1. Combining templates as needed
2. Templates that include essential and actionable information in the form of threat or element properties
3. More domain specific templates owned by specialists
4. Searching repositories of community provided templates and importing individual elements or threats

To further describe #1, let’s say you have a system with an IoT Bluetooth device that communicates to a mobile phone app and also has a cloud backend. Well, even if the backend was AWS, the previously mentioned AWS template wouldn’t be enough here. You’d want to combine the cloud template with other templates that have context for mobile and IoT applications.

For #2, this is still far off, and we need templates to catch on first. But, so far I have created a tool that is an experiment that aims to assist in deriving risk scoring metrics by extracting threat or element properties from a model file. The idea seems promising in semi-automating some metrics. And I still think it could be helpful when scoring a large amount of vulnerabilities. But it undoubtedly needs some fine tuning.

Besides determining risk scoring metrics, templates can hold the foundation for compliance standards such as CAPEC, CWE, and ATT&CK. pytm has a good threat library that experiments with this although as previously stated I don’t agree with threat model tooling providing templates or part of a template.

For #3, we need specialists sharing their templates for modelers to traverse domains with ease. Templates should be owned by companies, working groups, and community projects and open sourcing should be encouraged to get feedback from other specialists.

And last, for #4, we first need to solve #3, then organize those templates into searchable repositories or libraries that tooling can use. And possibly a standard format and syntax to increase interoperability.

Conclusion

Design and contribute templates. Use old models as a basis for doing so.

Here are some links if you want to explore Threat Model Templates:

• https://github.com/matthiasrohr/OTMT/blob/master/secodis%20web%20plain.tb7
• https://github.com/rhurlbut/CodeMash2020/blob/master/CodeMash2020-Default.tb7
• https://github.com/microsoft/threat-modeling-templates
• https://github.com/nccgroup/The_Automotive_Threat_Modeling_Template
• https://threatsmanager.com/downloads/templates/

Cybersecurity Risk

08 Apr 2022

I wanted to discuss some shower thoughts that I’ve had recently around how we perceive Cybersecurity risk. I recognize that the subject of cybersecurity risk is not very sexy. So being my first post, I gave the blog a HOT title.

First, if you haven’t already, check the whitepaper Towards Improving CVSS. TL;DR: of the paper is it identifies 3 critiques for CVSS to resemble Risk:

1. Failure to account for context (both technical and human-organizational).
2. Failure to account for material consequences of vulnerability (whether life or property is threatened)
3. Operational scoring problems (inconsistent or clumped scores, algorithm design quibbles)

You can also optionally check out A List of potential improvements for CVSS v4. We will cover some of these within this post but note that the v4 improvements contain the Towards Improving CVSS article referenced above.

I have scored with CVSS a bit but only started really digging into the issue after I stumbled on an interview with Art Manion, one of the original creators of CVSS. When asked how CVSS was formulated, what he had to say was quite alarming: “The math is sort of tortuous. There wasn’t a lot of transparency on that process. I’m on the SIG, so I actually know a little bit about what happened, but no one else really does. We didn’t even write it down that well internally, so it’s hard to tell.”

Wow! So answering questions like “Why is “Local” for Attack Vector equal to 0.71” is impossible to answer because no “one wrote it down”. So not only are people confusing CVSS for a real risk analysis, but we don’t have sound reasoning for it’s formula and values.

… Okay then what is risk? From a non-cybersecurity perspective, the simplest formula typically used is:

Risk = Impact x Likelihood

I want to acknowledge that everyone tends to define risk differently. Your CISO might think of impact in terms of Financial loss and QA might in terms of patient harm.

As a security professional, you might want to redefine the formula even further because you slightly read into it for a CISSP exam:

Risk = (Threat Actor Capability / Vulnerability Weakness) x Likelihood x Impact – Control Effectiveness

But at the end of the day

CVSS = Technical Severity

and

CVSS ≠ Risk

Or at least that’s how things currently stand…

Failure to Account for Context

This is a good point to remind you that the real world can be complex. So is assessing risk. Particularly because we are accounting for real world context and there are endless ways to do this.

CVSS Environmental v3.1 ONLY takes into account context of the user’s assets, that is it! We can take into account context for the asset’s mitigations in Modified metrics. We can also take into account for the importance of an asset, in terms of CIA, in Security Requirements. Therefore, environmental scoring gets slightly closer to the concept of risk by taking an asset’s context into the equation. But that’s not enough. Also keep in mind, CVSS was designed for assessing traditional IT assets, not everything under the sun.

Unfortunately, we don’t have context for how vulnerabilities can affect one another and be chained together. I imagine this would require a holistic view of all vulnerabilities in a system and mapping their complex relationships to things like ATT&CK patterns. There are vendors working hard on vulnerability chaining using ML/AI solutions with automated testing.

Next, we don’t have context of how the vulnerability is being used in the wild or if it’s actively being exploited. This could require gathering and utilizing threat intelligence, IoCs, classifying threat actors, or a Business Impact Analysis. This context could be considered either vulnerability factors (temporal) or business risk factors (environmental). Focus should be shifted to the skills and motivations of an attacker, and whether the effort required to exploit a vulnerability is less than the perceived gain the attacker will achieve by compromising the system rather than focusing on attempting to quantify likelihood of a future adverse impact in traditional probabilistic terms.

Last, we have no context of how the system is being used in the real world and a consistent context throughout a community. Take the risk cases of a hearing aid versus a pacemaker. They both have different context for their use cases. MITREs Medical Device Rubric for CVSS is an example of an attempt to harmonize context throughout an industry. But remember, since CVSS only has context at an asset level, we end up with a decent asset analysis but fail to take into account the device use case and other environmental factors which will tie into #2.

Failure to Account for Consequences Either Life or Property

CVSS v4 has purposed to add a safety metric. So this is a big win for accounting for patient harm and loss of life. Beyond medical, automotive, Operational Technology, and aerospace industries could benefit from this metric.

Although, a quick search of the word “property” in the CVSS CIGs working group’s “v4 Working Items” resulted in 0 results. Not sure if this is to be considered but looks unlikely.

Operational Scoring Problems

To put it nicely, the CVSS formula was pulled from where the sun doesn’t shine. There was never any solid empirical rational provided for the structure of the formula or for the constant values of the formula weights. The paper calls for a complete overhaul in the formula and goes over an effective approach to do so.

Likelihood

Let’s dive into likelihood. This is typically the more controversial and political aspect of the traditional risk formula.

“What does likelihood mean, specifically for Cybersecurity risk? How can we define it?”

Definitions

• ISO 31000 defines likelihood as:“the chance of something happening.”
• NIST 800-30: “the probability that a given threat is capable of exploiting a given vulnerability or a set of vulnerabilities.”
• AAMI TIR57 Risk Management Principles for Medical Device Security & CNSSI No. 4009 defines likelihood of occurrence as: “weighted factor based on a subjective analysis of the probability that a given threat is capable of exploiting a given vulnerability”

Safety risk management is designed to deal with failures in which the likelihood is based on design and manufacturing factors. Whereas security risk management likelihood is more of an estimate to whether an attacker will invest time and resources to exploit that vulnerability.

Cybersecurity risk, in particular, needs to deal with the case of the black swan one-time events. These events have high impacts and low likelihood. Such events could wipe out a company and be devastating.

Likelihood, in the case of Cybersecurity, therefore needs to reflect the likelihood, or probability, that a one-time event could occur. Likelihood to affect the organization once in the entire product lifetime. Likelihood to achieve the highest protection against evolving threats.

Threat actors exploit vulnerabilities with particular goals in mind and not simply because a vulnerability exists with a high CVSS score. So how can we gauge the likelihood of a one time event?

Context is key.

Quick Recap:

If the attack is theoretical and not proven, this lowers the likelihood. If we have available mitigations to directly address the vulnerability, this lowers the likelihood. If we are patching software and managing risk in a mature manner, this lowers the likelihood. If the CVE is not actively being exploited by APTs in the wild, this lowers the likelihood.

Impact

While not as controversial as likelihood, I believe impact can have just as many issues.

Let’s start we the fact that we are using CIA as a means to gauge impact. The lack of granularity in just high, medium, and low metrics can present a challenge for achieving accurate CIA values.

Next, the fact that every company has different operating factors, and maturity levels, and their own interpretations can lead to a different risk environment at the organizational level.

Now let’s look at the CVSS Environmental formula to see how Security Requirements are used: ** 1 - [ (1 - ConfidentialityRequirement × ModifiedConfidentiality) × (1 - IntegrityRequirement × ModifiedIntegrity) × (1 - AvailabilityRequirement × ModifiedAvailability) ]**

Unfortunately, with Security Requirements are only gauging impact at the asset level, we have no context of vendor or regulatory risk levels. If the vendor only designs class 3 implants, it would probably have a high organizational and regulatory risk. If regulatory had strict requirements, this would be high regulatory risk. Ultimately this is NOT the vendors decision.

On top of all that, we also lack ways to determine Security Requirements based on Threat Actor Risk in a given environment.

So perhaps we can say an asset’s individual impact is something like:

(IR x CR x AR) = Asset Impact

And when we combine all of the new environmental impacts in a some meaningful way we get:

(Asset Impact + Organizational Impact + Regulatory Impact + Threat Impact) = Environmental Impact

This new Environmental impact would be more encompassing beyond just the individual asset level. I suggest we focus on creating repeatable frameworks for deriving Security Requirements so that companies can factor this into their risk analysis.

Other Scoring Specs

EPSS takes complexity and how the vulnerability is being exploited into account. So does VPR, which is maintained by tenable. I personally like VPR.

Metrics in VPR are: Vulnerability Age, CVSS Impact, Exploit Code Maturity (CVSS), Product Coverage (similar to target distribution), Threat Sources, Threat Intensity (or frequency), and Threat Recency (in days)

SSVC is another decent risk scoring spec. CVSS’s mathematical inputs are vectors which have a direction and magnitude. The formula gets evaluated with curve fitting math which outputs a reduced partial range. SSVC uses decision points as inputs and evaluates with decision trees that outputs a qualified priority. Decision trees are qualitative actions that an organization can take. SSVC uses 6 decision point categories: exploitation (threat feed), technical impact (cvss base), utility, exposure, mission impact, and safety impact. Exposure, mission impact, and safety impact can all be asset management.

CVSS v4

Let’s dig into CVSS v4 working items.

Threat Metric Group

Temporal Metric Group is replaced with Threat Metric Group as the SIG focuses on the importance of using Threat Intelligence for accurate scoring. An additional Threat Intelligence metric has been purposed. On top of that, there is pending removal of Report Confidence and Remediation Level metrics. Any previous temporal exploitability metrics should go into base.

Environmental

Representation of vendor-supplied Severity/Impact scoring within CVSS standard. This is interesting because Security requirements reflect asset importance, whereas vendor-supplied Severity reflects the vendor’s risk level. Lastly, we would still need an industry or regulatory defined risk level to get full context of Environmental Impacts.

Target distribution to be added back. Similarly used in such scoring metrics like Vulnerability Priority Rating (VPR).

The working items mentions the “Towards Improving CVSS.” whitepaper at the end and acknowledges a shift needed to focus on deciding response priority, which is more like risk than severity.