The Cybersecurity Maturity Matrix

I was introduced to the Cyber Defense Matrix circa 2015, shortly after it was released by Sounil Yu. The framework is based on two dimensions: Asset classes, or things you want to defend. And the NIST Five Functions, activities performed to defend assets, which range from identification through post-incident recovery.

I’ve always found this framework simple, adaptable, and useful for a wide variety of purposes:

• As a organizational security leader, to catalog the organization’s cyber and information security controls
• As a vendor, to map customers’ capabilities, and to illustrate where and how our own security solutions contribute to their program
• As a security product leader, to visualize our current strengths, and to chart a path into new areas

No matter the use case, it’s an invaluable tool for discussion and alignment related to security controls, operations, maturity, products, and more. And for all of the times I’ve found myself remaking this matrix on mediums ranging from slide decks to paper flip charts, I’ve always wanted a tool to make it easier to get started, and easier to save our work.

The Cybersecurity Maturity Matrix is a simple tool for visualizing maturity, control coverage, and other aspects of your security program. It’s based on the Cyber Defense Matrix, and provides the following features:

• Customizable asset classes: Start with the default set, or the original Cyber Defense Matrix asset classes, and then make any changes that you wish.
• Four maturity levels: Each cell can be left empty or assigned one of three states:
• Optional cell annotations: Use annotations to list products, responsible individuals or teams, or dates when you expect to reach a milestone.
• Export to PNG: Drop the completed matrix into your doc or presentation in seconds.
• JSON export and import: Export the details of your completed matrix to JSON, so that you can import and update it whenever you wish.

Try it now: https://cybermaturitymatrix.com

Something else you’d like to see? Drop me a line: cmm-feedback @ this domain.

Trust and the cybersecurity buyer’s journey

The buyer’s journey is a common framework for understanding how customers (who have problems) find products provided by vendors (to address problems).

• Awareness - Knowing that a problem and potential solutions exist. Brand awareness is a key component of this stage.
• Consideration - Which potential solutions are you considering?
• Evaluation - Of the solutions being considered, how do they stack up? Feature comparisons, pros and cons, and other criteria are examined.
• Purchase - A solution is chosen and purchased. This stage is also referred to as “Adopted.”
• Post-Purchase - The solution is implemented. Critically, it’s here that the customer makes some early and important determinations with regards to how the solution performs relative to their expectations and the vendor’s promises during Evaluation.

What’s unique about the security industry?

Put simply: The security industry is grounded in the principle of trust:

• Trust that the team has a deep understanding of the problem, credibility in the space, and acts with integrity.

When companies purchase virtually any other type of technology, they tend to seek out specific functionality, can easily determine whether the product works as intended, and may have an interest in the roadmap but aren’t dependent on future functionality for their success.

When we purchase security products—particularly those that we rely on to perform critical operational functions—our journey is unique:

• We want some specific functionality (e.g., reporting, interoperability), but what we really want is an outcome (e.g., a net reduction in harmful emails delivered to users).
• While we typically have some tests that we can use to validate that the system works as intended, we can be assured that the system will face an endless barrage of inputs designed by a capable actor to avoid detection.
• We often have a keen interest in the vision and roadmap as a bellwether for the team’s depth of understanding, commitment, and ability to productize or operationalize threat intelligence findings (i.e., as adversaries evolve, will the product evolve to meet them, and do so in a timely manner?).

Major features we’ll try to discern early on by way of the website, webinars, and other public materials. Efficacy we can test superficially during an evaluation. But oftentimes we won’t get to Evaluation (or even Consideration) unless we have a good sense for whether the team understands the problem and has credibility in the space—key ingredients of trust. Most of the time, we’ll have a sense for this as (or before) we begin this journey, by virtue of personal or professional networks, conferences, and published research.

Also note that trust in the team can absolutely compensate for lack of features, or even some lack of efficacy. Cybersecurity is an adversarial discipline, and great products work a great percentage of the time, but nothing’s perfect. A trustworthy and capable team will shine in the face of failure, responding with transparency and decisive action.

This concept is important to leaders because trust, inclusive of credibility and integrity, are virtually impossible to fake or decide on a whim that you want. The behaviors and principles that lead to trust either come naturally to founders and early employees, and are integral to company culture, or they are not. And ultimately, despite how nice your product and marketing look, security practitioners are looking through these things to determine whether and to what degree they trust your team to deliver on an ongoing basis against a determined and skilled adversary.

Improving CISA KEV data integrity

tl;dr - Sometimes CISA removes Known Exploited Vulnerabilities catalog records. It would be awesome if they marked them as withdrawn and/or superseded instead.

Background

The Cybersecurity and Infrastructure Security Agency (CISA) Known Exploited Vulnerabilities (KEV) catalog has fast become an integral source of vulnerability data consumed by organizations, products, researchers, and notably entities subject to CISA’s mandate to reduce the risk of known exploited vulnerabilities. Widespread adoption of KEV is a key component of the program’s charter:

The KEV catalog sends a clear message to all organizations to prioritize remediation efforts on the subset of vulnerabilities that are causing immediate harm based on adversary activity. Organizations should use the KEV catalog as an input to their vulnerability management prioritization framework . . . Organizations should also consider using automated vulnerability and patch management tools that automatically incorporate and flag or prioritize KEV vulnerabilities.

As the CISA KEV catalog grows in terms of both data and adoption, it’s important that the schema and data support core use cases, including basic analytics and automation.

CISA KEV criteria and data structure

To be included in the catalog, a vulnerability must have a CVE ID assigned, be under active exploitation, and CISA must be able to provide remediation guidance (more on these requirements here).

Once admitted into the catalog, an entry is added based on the published schema. At the time of writing, the KEV vulnerability data structure contains the following keys:

vendorProject 
product 
vulnerabilityName 
dateAdded 
shortDescription 
requiredAction 
dueDate 
knownRansomwareCampaignUse * 
notes * 

* Not required

So, the KEV catalog is a structured data set that is the product of cyber threat intelligence collection and analysis. Structured data ensures predictable outputs, and is meant to enable ingestion into other systems, ranging from spreadsheets to vulnerability management products and frameworks. But “intelligence collection and analysis” implies that, as new information becomes available, the data set may be subject to substantive updates.

Where things break down

As CISA’s understanding of a given vulnerability changes, it may no longer make sense for the vulnerability to remain in the KEV catalog. When that happens, CISA typically does two things:

1. The vulnerability is removed from the catalog.
2. Another related vulnerability may be added or, if a related vulnerability is present, its shortDescription field is updated with a note that looks like this:

A consequence of this is that, when the catalog is exported via JSON or CSV, we cannot assume that data prior to the time of export is still valid. So, some amount of reconciliation needs to take place. Here’s how the above change looks when exporting the catalog via CSV:

The most recent entries as of 2023-10-20:

The most recent entries as of 2023-10-30:

Note that the initial entry for CVE-2021-1435 on row 1023 is no longer present on 2023-10-30, and a new entry for CVE-2023-20273 appears on that row instead (that both entries on row 1023 are related is coincidental).

This happens from time to time, and as you can see in the more detailed screenshot above, CISA does make an effort to reference related vulnerabilities in the shortDescription field. However, these notations are part of unstructured text, and make it challenging to consume and reconcile the KEV catalog on an ongoing or automated basis without error-prone parsing of these description fields (which we cannot assume will always reference related vulnerabilities in a predictable manner, or at all).

Proposal: dateWithdrawn and supersededBy

Once a vulnerability is added to the KEV catalog, it is never removed but can be marked as “withdrawn”.

dateWithdrawn - The date the vulnerability was withdrawn as a catalog entry, in the format YYYY-MM-DD

If a vulnerability is superseded by another (e.g., as CVE-2021-1435 was superseded by CVE-2023-20273), that relationship is made explicit:

supersededBy - The CVE ID that supersedes this vulnerability

Thank you, CISA!

I’ve written and said many times how valuable a service the KEV catalog is to both the public sector and to private industry. The above is not a complaint—far from it.

Making the above changes (or something like them) would improve the integrity of the catalog, ensure that vulnerability records within the catalog are preserved, and that consumers of the catalog (human or computer) aren’t burdened with any ambiguity or reconciliation tasks.

When your product becomes a feature

Having spent a few years using, maintaining, and building security products of every conceivable shape and size, it’s become apparent how uniquely risky it is to invest in building cloud[1] security products.

There are good reasons to build a cloud security product:

• Every aspect of cloud adoption is growing at a meaningful clip and shows no signs of slowing. “Cloud customers” as an addressable market is huge.
• Cloud products are numerous, can change at any time, and are thus easy to adopt but hard to master.
• Cloud security concerns are very real, and cloud customers are willing to spend money on cloud security products to reduce risk.

But there are very real risks from a vendor (particularly startup) standpoint:

• Because cloud security concerns are very real and cloud customers are having to spend additional money on point cloud security products to reduce risk, they are rightfully pressuring cloud vendors to improve the overall security posture of their products.
• A cloud security product cannot, in general, take any preventive or corrective action that the cloud product itself does not support.
• The cloud product vendor can, by definition, see everything that both the end customer and the cloud security product are doing (either directly or via inference).

The net of the above is that, when any cloud security problem becomes enough of a business risk to the cloud product vendor, the cloud product vendor can simply make that problem go away.

The same is true for entire classes of attacks, which a standalone product or team can focus on and solve by playing whack-a-mole, but that the vendor can oftentimes address by making the class of attack go away. Good examples of this include things like secure defaults for cloud-based storage (make the “oops I put it on the Internet” problem go away) and default multi factor authentication (make the “I got phished” problem go away).

Vulnerability half-life and the cloud

The half-life of security vulnerabilities affecting cloud-based products is nowhere near what we’re accustomed to based on decades of complex, multipurpose, customer-managed software. When building for the latter, there will always be enough attack surface and enough configuration and/or version drift to justify relatively expensive security solutions. This means that vendors building security products can expect to be paid back on their investment, and for a meaningful amount of time.

Cloud security problems only live as long as the cloud product vendor wants them to live. And as a result of (healthy) customer expectations—e.g., better security defaults—we can expect both software and configuration vulnerabilities in cloud products to be addressed with increasing efficiency and effectiveness.

Upstreaming of controls

The most effective security controls become features of the platforms they were designed to protect. Years ago, host-based firewall and application control were significant product niches. Over time, they became flagship features of larger third-party security product suites. Today, we can’t imagine an end-user operating system that doesn’t include both of these controls.

Cloud security products are no different. But unlike consumer or end user devices, where solutions may exist but are optional and/or won’t be adopted overnight, solutions to cloud security problems are oftentimes not optional and are literally “adopted” overnight.

The net of this is that many cloud security products run the very real risk of becoming obsolete, sometimes overnight and with little warning. Early entrants into the cloud posture management space felt this, as Amazon and Microsoft moved aggressively into a mix of stronger default posture, log and alert aggregation, and more.

It’s not all doom and gloom

The beauty of cloud products, including cloud security products, is that it’s far easier to build, maintain, and support than on-premise software. So, the cost to prototype, deploy, and support cloud security products may be lower than other types of security products (particularly endpoint- or device-based products, which can be notoriously high friction).

I also said that many cloud security products run the very real risk of becoming obsolete. There are plenty of cloud security needs that are much more durable than wrappers around eventually-native technical controls. In most cases, these will be cloud-optimized products that address evergreen problems, many of them operational. A few examples include:

• Log aggregation and analytics
• Threat detection, investigation, and response
• Incident management

Still, security startups must be thoughtful about the level of investment they’re willing to make and how much of a moat they can establish, understanding that cloud security problems in particular are almost always addressable upstream, and will eventually be addressed upstream.

[1] Cloud is defined for the purpose of this article as the combination of infrastructure- (IaaS), platform- (PaaS), and software-as-a-service (SaaS).

Top initial access techniques from 2019-2022, mapped to ATT&CK

Based on initial access data from dozens of cybersecurity industry reports over the past four years, here’s a visualization of the top five initial access techniques, and how they’ve trended over the years:

More on some of the source data and the top initial access techniques from 2022 can be here.

The data

Rankings by year, used to generate the visualization above.