💻 computer science

Assessing the ROI of Cyber Threat Intelligence: An Operational and Financial Evaluation Framework

This paper proposes a comprehensive framework for assessing the Return on Investment of Cyber Threat Intelligence by introducing a Threat Intelligence Effectiveness Index (TIEI) to measure operational maturity and a breakeven-first financial method to quantify investment viability despite the inherent difficulty of measuring prevented cyber incidents.

Original authors: Matteo Strada, Stelvio Cimato

Published 2026-07-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Matteo Strada, Stelvio Cimato

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a massive, high-tech spaceship cruising through the asteroid belt of the internet. Your job is to keep the ship safe from space pirates, rogue asteroids, and sneaky saboteurs. You have a team of lookouts, a radar system, and a shield generator. But here's the tricky part: the best day your team can have is a day where nothing happens. No alarms, no explosions, no damage. The ship just keeps sailing smoothly.

This is the world of Cyber Threat Intelligence (CTI). Think of CTI as the "super-spy network" for your digital ship. Instead of just waiting for an alarm to go off, CTI gathers secret reports from other ships, spies, and sensors to tell you exactly where the pirates are hiding, what kind of weapons they are using, and how they plan to attack. It turns a mountain of confusing noise into a clear, actionable map.

But there's a huge problem for the people paying for this spy network (the ship's owners, or in the real world, company bosses and boards). They want to know: "Is this expensive spy network actually worth the money?" In normal business, if you buy a new engine and the ship goes faster, you can measure the speed. But in cybersecurity, if you buy a spy network and the ship doesn't get attacked, how do you prove you saved money? You can't show a receipt for a disaster that never happened. This is the "Prevention Paradox": the better you are at your job, the less visible your success becomes.

This is where a new study by Matteo Strada and Stelvio Cimato comes in. They are like the ship's engineers trying to build a new kind of calculator. They know you can't just guess the value of a "non-event," so they created a clever two-part system to figure out if your spy network is actually saving you cash.

First, they built a "Threat Intelligence Effectiveness Index" (TIEI). Imagine this as a "Spy School Report Card." Instead of just giving a single grade, it checks four different skills:

Quality: Are the spy reports actually true and useful?
Enrichment: Did the team add extra details to the reports (like connecting a pirate ship to a specific gang)?
Integration: Did the team actually feed these reports into the ship's radar and shields, or did they just sit in a drawer?
Impact: Did the ship react faster? Did the shields block more attacks?

The cool thing about their report card is that it's strict. If your team is amazing at getting reports but terrible at using them, your overall grade tanks. It's like a chain: the whole chain is only as strong as its weakest link. You can't have a perfect score if one part is failing.

Second, they created a "Break-Even Map." Since no one knows exactly how many times a pirate would have attacked if they didn't have the spy network, the authors say: "Let's stop guessing and start mapping." They drew a line on a graph that shows the relationship between two unknowns:

How likely is a big attack? (The probability).
How much does the spy network stop it? (The mitigation).

The map shows that for your spy network to be worth the cost, these two numbers must multiply to be bigger than a specific "tipping point." For example, in the Finance sector (banks), the study illustrates that if your spy network can stop 20% of potential disasters, your company would need to face a realistic chance of a major attack of at least 34.4% per year to break even. In Healthcare (hospitals), the numbers are similar: you would need a 34.9% chance of a big attack to justify a 20% stop-rate.

Crucially, the authors did not "find" these numbers as universal facts. They are not predictions of what will happen. Instead, they are illustrative stress-test inputs. The study explicitly states that these are hypothetical scenarios used to show how the math works. The numbers change depending on your specific organization's costs and risks.

The authors also ran thousands of computer simulations (like running a video game a million times with different settings) to see how the math holds up under uncertainty. In these simulations, they used the "Spy School Report Card" (TIEI) only as a calibration assumption for a specific reference scenario. They did not conclude that a high TIEI guarantees you will reach break-even in the real world. Instead, they showed that if you assume a mature program (high TIEI), the math suggests a wider range of plausible success rates. If your program is less mature, the range of plausible success rates shrinks.

The big takeaway isn't that they found a magic number that proves CTI is always a winner. Instead, they gave companies a honest tool to stop guessing. They say, "Don't just tell your boss 'we saved you millions' because you can't prove it. Instead, show them your Report Card to prove you are good at your job, and then show them this Map to prove that, given your specific risks and the assumptions you are willing to make, the investment makes sense."

It's a way to turn the invisible art of "preventing bad things" into a clear, logical business decision. It admits that we can't know the future, but it gives us a solid way to plan for it, ensuring that the money spent on spies is actually protecting the ship, rather than just buying expensive, unused maps.

Technical Summary: Assessing the ROI of Cyber Threat Intelligence

1. Problem Statement

The quantification of Return on Investment (ROI) for Cyber Threat Intelligence (CTI) programs faces a fundamental measurement problem: successful prevention results in "non-events" that leave no observable financial signal. Consequently, CTI spending resists traditional cost-benefit analysis. Furthermore, CTI's value is often indirect, acting as a force multiplier for downstream controls (e.g., firewalls, EDR, analysts) rather than a standalone technical control. This complicates causal attribution and the definition of the investment boundary. Traditional economic models, such as the Gordon–Loeb (GL) model, are designed for independent technical controls and fail to capture the probabilistic risk-reduction and leverage effects inherent to intelligence-driven workflows. Without a defensible model, CTI programs risk being perceived as cost centers, leading to underfunding and misallocation of resources.

2. Methodology

The paper proposes a multi-layered framework that combines quantitative metrics, qualitative indicators, and a specific financial modeling approach to address the measurement gap.

2.1 The Threat Intelligence Effectiveness Index (TIEI)

To operationalize CTI maturity, the authors introduce the Threat Intelligence Effectiveness Index (TIEI). Unlike traditional weighted arithmetic means, which assume linearity and allow high scores in one area to compensate for failures in another, TIEI utilizes a weighted geometric mean. This formulation penalizes weak links, reflecting the reality that a deficiency in one dimension (e.g., poor integration) can negate the value of others.

The TIEI aggregates four dimensions:

Intelligence Quality: Accuracy, timeliness, relevance, and completeness.
Enrichment: Mapping to TTPs (ATT&CK), internal correlation, entity linking, and actionability.
Integration & Automation: Control coverage, delivery success, automation rates, and SLA adherence.
Operational Impact: Reductions in Mean Time to Detect (MTTD) and Mean Time to Respond (MTTR), false-positive reduction, and coverage lift.

Scores are normalized to a [1, 100] scale using defined targets or multi-anchor normalization (unacceptable, reference, stretch levels) to ensure reproducibility.

2.2 Financial Framework: Breakeven-First Approach

The paper shifts from predicting specific ROI values to defining a breakeven feasibility frontier. This approach acknowledges that the organization-specific annual probability of a material loss event ( $p_0$ ) and the CTI-attributable mitigation fraction ( $M$ ) are unknowns that cannot be reliably estimated from broad sector prevalence reports.

The model defines the Total Cost of Ownership (TCO) for CTI ( $TCO_{CTI}$ ) to include not just platform and feed costs, but also the incremental downstream costs required to operationalize intelligence (integration, engineering, automation).

The condition for positive ROI is derived as:
$p_0 \times M > \kappa$
Where $\kappa = \frac{TCO_{CTI}}{L_M}$ , and $L_M$ is the conditional loss magnitude of the event.

This creates a two-unknown frontier: an organization must substantiate that the product of its specific event probability and its specific mitigation capability exceeds the cost-to-loss ratio.

2.3 Conditional Simulation

To illustrate how uncertainty propagates, the authors employ a PERT simulation conditioned on the TIEI score. The mitigation factor ( $M$ ) is modeled as a PERT distribution where the bounds (pessimistic, most likely, optimistic) are affine functions of the TIEI score. This links operational maturity directly to the range of plausible financial outcomes without asserting a single deterministic value.

3. Key Contributions

The paper makes two primary contributions:

The Threat Intelligence Effectiveness Index (TIEI): A structured, weighted geometric maturity measure that captures intelligence quality, enrichment, integration, and operational impact. It explicitly penalizes unbalanced performance and provides a reproducible basis for tracking program evolution.
A Breakeven-First Financial Decision Framework: A method that treats $p_0$ and $M$ as unknowns, characterizing the combinations required for positive ROI. This framework includes a comprehensive definition of the investment boundary (TCO) and demonstrates how to derive scope-matched breakeven requirements without relying on misleading sector-wide prevalence data.

4. Results and Application

The framework was applied to illustrative scenarios in the financial services and healthcare sectors.

Financial Services: Using an estimated breach cost ( $L_M$ ) of $5.82M and an all-in annual investment ( $TCO_{CTI}$ ) of $0.40M, the feasibility constant $\kappa$ is 6.87%. The analysis shows that for a CTI program to break even with a 20% mitigation capability ( $M=0.20$ ), the organization's scope-matched annual probability of a material event must exceed 34.4%.
Healthcare: With a higher estimated breach cost of \ $8.595M and an investment of \$ 0.60M, $\kappa$ is 6.98%. Similarly, a 20% mitigation capability requires an event probability exceeding 34.9%.

The paper emphasizes that these are not predictions of sector risk but feasibility frontiers. They define the minimum evidence an organization must provide to justify the investment. The conditional PERT simulation further illustrates that for a mature program (TIEI = 80), the probability of achieving positive ROI increases significantly as the assumed event probability ( $p_0$ ) rises, but the simulation relies on the organization supplying a defensible $p_0$ .

5. Significance and Claims

The authors claim that this framework provides a reproducible basis for CTI investment decisions by linking operational maturity (TIEI) to an auditable financial boundary.

Reframing ROI: The paper argues that ROI should be viewed as a measure of "fragility avoided" rather than profit gained, addressing the "prevention paradox" where successful security is invisible.
Causal Consistency: By refusing to substitute broad sector prevalence for organization-specific event probabilities, the framework avoids the "category error" of applying traditional control-based models to informational assets.
Holistic Valuation: The approach integrates quantitative performance indicators (MTTD, MTTR), qualitative strategic value (brand protection, regulatory compliance), and financial modeling (FAIR-based risk reduction).
Modesty in Claims: The paper explicitly states that it does not estimate a universal sector ROI. The numerical frontiers are deterministic sensitivity bounds based on scenario inputs, not confidence intervals. The TIEI-conditioned PERT simulation is presented as a secondary, assumption-conditional illustration rather than a predictive tool.

The study concludes that while significant measurement challenges exist (attribution, data overload, intangibles), a multi-layered approach combining the TIEI with a breakeven-first financial model allows security leaders to communicate CTI value as a defensible strategic investment rather than a cost center.