MCP-in-SoS: Risk assessment framework for open-source MCP servers

This paper addresses the lack of systematic security evaluation for open-source Model Context Protocol (MCP) servers by applying static code analysis to identify Common Weakness Enumeration (CWE) vulnerabilities, mapping them to MITRE CAPEC attack patterns, and introducing a multi-metric risk-assessment framework to guide secure-by-design development.

The Gist

Here is an explanation of the paper "MCP-in-SoS: Risk assessment framework for open-source MCP servers" using simple language and creative analogies.

🌟 The Big Picture: The "Universal Remote" Problem

Imagine you have a super-smart robot assistant (an AI) that can do anything: write code, book flights, or control your smart home. To do these things, the robot needs to talk to different tools.

MCP (Model Context Protocol) is like a universal remote control invented recently. It lets the robot talk to any tool (like a calculator, a database, or a file system) using the same simple language.

Because this universal remote is so easy to use, thousands of people have started building their own "tool adapters" (called MCP Servers) and posting them online for free. It's like a massive, open-source app store where anyone can upload a plugin.

The Problem: Just because these adapters are free and easy to get doesn't mean they are safe. Some might have broken locks, open windows, or hidden backdoors. If a hacker finds a bad adapter, they can trick the robot into stealing your secrets, deleting your files, or taking over your computer.

🔍 What Did the Researchers Do?

The researchers from New Mexico State University and the University of Hartford decided to play "security inspector" for this new ecosystem. They didn't just guess; they built a systematic inspection machine called MCP-in-SoS.

Think of it like a high-tech car safety inspector that scans thousands of used cars (the open-source servers) to find hidden mechanical failures before you buy one.

The 4-Step Inspection Process:

1. The Scan (Static Analysis):
   They used automated tools (like CodeQL and Joern) to read the code of 222 different MCP servers. Imagine a robot reading the blueprints of 222 houses to find cracks in the foundation, unlocked doors, or exposed wiring.

   • Result: They found 15,962 potential flaws.

2. The Translation (Mapping to Standards):
   They took the messy list of errors they found and translated them into a standard "safety language" called CWE (Common Weakness Enumeration).

   • Analogy: Instead of saying "the door handle is wobbly," they categorize it as "CWE-306: Missing Authentication" (No lock on the door).

3. The Threat Match (Connecting to Attackers):
   They then looked up how hackers actually use these specific flaws. They used a database called CAPEC (Common Attack Pattern Enumeration and Classifications).

   • Analogy: They matched "Missing Lock" with "Theft via Open Door." This helps them understand not just what is broken, but how a criminal would exploit it.

4. The Risk Score (The Report Card):
   Finally, they created a scoring system. They asked: "How likely is a hacker to break this?" and "How bad would it be if they did?"

   • They gave each server a risk score from Very Low to Very High.

📊 What Did They Find? (The Shocking Results)

The inspection revealed some scary truths about the current state of these tools:

• Most Are Flawed: Out of the 222 servers they checked, 86% (191 of them) had at least one serious weakness. It's like checking 100 used cars and finding that 86 of them have a flat tire or a broken brake.
• High Risk is Common: Nearly 66% of the servers were rated as High or Very High risk.
• The "Chain Reaction" Effect: The most dangerous finding wasn't just one broken part; it was how they connect.
  • The Analogy: Imagine a house where the front door is unlocked (Protocol weakness). Because the door is open, the thief can easily walk into the kitchen (Tool weakness) and steal the silverware (Resource weakness).
  • The researchers found that weak access controls (unlocked doors) almost always appear alongside tool injection flaws (broken kitchen counters). This creates a "multi-stage exploit chain" where one small mistake leads to a total disaster.

🏗️ The Four "Threat Surfaces"

The researchers categorized where the problems usually hide:

1. Protocol (The Front Door): 57% of all problems were here. This is the main entry point. If the protocol is weak, the whole house is vulnerable.
2. Resource (The Safe): 29% of problems. This involves leaking sensitive data like passwords or private files.
3. Tool (The Appliances): 10% of problems. This is where the robot tries to use a tool but gets tricked into doing something malicious.
4. Prompt (The Instructions): Only 4% of problems found here, but very dangerous. This is when a hacker tricks the robot's brain into giving bad orders.

💡 The Takeaway

The paper concludes that while the Model Context Protocol is a brilliant idea for connecting AI to the real world, the open-source ecosystem is currently a "Wild West."

Developers are building these tools quickly, but they aren't building them securely. The researchers are calling for a "Secure-by-Design" approach. Before we let AI agents run our banks, hospitals, or homes, we need to ensure the "universal remote" adapters they use are built with strong locks, not just duct tape.

In short: The technology is amazing, but the security is currently "under construction," and we need to fix the foundation before we invite the AI in for dinner.

Technical Summary

Here is a detailed technical summary of the paper "MCP-in-SoS: Risk assessment framework for open-source MCP servers."

1. Problem Statement

The Model Context Protocol (MCP), introduced by Anthropic in late 2024, has rapidly become the standard for connecting Large Language Model (LLM) agents to external tools and data sources. While MCP offers a clean separation of concerns and improved portability compared to ad-hoc integrations, it fundamentally alters the security posture of agentic systems. Unlike traditional static client-server applications, MCP systems feature dynamic runtime control flows where agents can chain tool calls and trigger server-side actions that reach sensitive resources (filesystems, APIs, credentials).

Despite the proliferation of open-source MCP servers, there is a critical gap in understanding the systematic security risks inherent in their code. Existing research has focused on theoretical threat taxonomies or constructing malicious servers for attack simulation, but no prior study has conducted a large-scale, empirical assessment of implementation-level weaknesses (code-level vulnerabilities) in publicly available MCP servers. This lack of data hinders the development of secure-by-design practices for the growing MCP ecosystem.

2. Methodology: The MCP-in-SoS Framework

The authors propose MCP-in-SoS, a structured, four-stage automated pipeline designed to detect, normalize, and score security weaknesses in MCP server implementations.

A. Data Collection

• Dataset: 222 publicly available Python MCP server repositories from GitHub (filtered by stars >100 and active development).
• Exclusions: Client-only projects, benchmarks, and intentionally vulnerable repositories.

B. The Four-Stage Pipeline

1. Static Code Analysis:
   • The pipeline employs three distinct analyzers to extract candidate weaknesses:
     • CodeQL: For high-precision identification of known patterns (e.g., injection sinks).
     • Joern: Used to generate a Code Property Graph (CPG) and execute custom queries aligned with the 2025 CWE Top 25. The authors developed 54 specific Joern queries using LLM assistance.
     • Cisco AI Defender MCP Scanner: Used on a subset to detect malicious patterns via rule-based signatures and LLM-as-a-judge.
2. Metadata Preparation:
   • Ingests the MITRE CWE (Common Weakness Enumeration) and CAPEC (Common Attack Pattern Enumeration and Classification) databases.
   • Links CWE weaknesses to CAPEC attack patterns.
   • Handles missing data (e.g., missing "Likelihood of Exploit" in CWE) by imputing values from parent categories or related CAPEC entries.
3. Normalization:
   • Deduplicates findings across tools.
   • Maps all tool outputs to standardized CWE identifiers.
   • Aggregates findings per repository to calculate occurrence frequencies.
4. Risk Scoring:
   • Risk Index Calculation: Uses a metadata-driven model based on $Risk = Likelihood \times Impact$.
     • Likelihood: Derived from the product of CAPEC Likelihood of Attack, CWE Likelihood of Exploit, and the count of Modes of Introduction.
     • Impact: Derived from the product of CAPEC Typical Severity and the count of Common Consequences.
   • Repository Scoring: Computes a repository-level risk score ($R_{overall}$) by combining a severity profile (Root-Mean-Square of weighted CWE risks) with a log-scaled finding volume factor to prevent outliers from dominating the scale.

C. Threat Surface Taxonomy

The framework categorizes weaknesses into four MCP-specific threat surfaces:

1. Protocol: Transport and access control issues.
2. Tool: Vulnerabilities in tool handlers and invocation logic.
3. Resource: Issues regarding data exposure and configuration.
4. Prompt: Vulnerabilities related to prompt injection and confused-deputy behaviors.

3. Key Contributions

• Large-Scale Assessment: The first systematic analysis of 222 open-source MCP servers, identifying 15,962 total findings across 51 distinct CWE classes.
• Reproducible Pipeline: A novel workflow combining static analyzers (CodeQL, Joern) with LLM-assisted query generation and CWE-CAPEC mapping.
• Custom Query Suite: A set of 54 Joern queries specifically aligned with the 2025 CWE Top 25 for Python MCP servers.
• Risk Scoring Model: A novel metric that integrates CWE and CAPEC metadata to produce both CWE-level and repository-level risk scores.
• Conditional Co-occurrence Analysis: An empirical study revealing how different threat surfaces combine to form multi-stage exploit chains.

4. Key Results

• Prevalence of Weaknesses: 86.0% (191 out of 222) of the analyzed repositories contain at least one mapped security weakness.
• Risk Distribution:
  • 47.6% of repositories fall into the High risk band.
  • 18.3% fall into the Very High risk band.
  • Only 0.5% are classified as Very Low risk.
• Dominant Weaknesses: The top 5 CWEs account for 75.7% of all findings:
  1. CWE-862 (Missing Authorization)
  2. CWE-200 (Exposure of Sensitive Information)
  3. CWE-306 (Missing Authentication)
  4. CWE-287 (Improper Authentication)
  5. CWE-89 (SQL Injection) – Notable for being both frequent and high-risk.
• Threat Surface Analysis:
  • Protocol weaknesses are the most prevalent (56.9% of findings) and contribute the most to risk exposure (57.1%).
  • Tool weaknesses, while less common (9.9%), contribute disproportionately to risk (21.2%) due to their high impact potential.
• Exploit Chains (Conditional Co-occurrence):
  • Weaknesses rarely exist in isolation.
  • 87% of repositories with Tool weaknesses also have Protocol weaknesses.
  • 88% of repositories with Resource weaknesses also have Protocol weaknesses.
  • 94% of repositories with Prompt weaknesses also have Resource weaknesses.
  • Conclusion: Protocol weaknesses often act as an "enabling layer" that allows attackers to reach and exploit downstream Tool and Resource vulnerabilities.

5. Significance and Implications

• Security-by-Design Urgency: The findings demonstrate that the current open-source MCP ecosystem is rife with exploitable vulnerabilities that could compromise confidentiality, integrity, and availability. This underscores the critical need for secure development practices in MCP server creation.
• Shift in Attack Surface: The study highlights that MCP security is not just about the LLM or the prompt, but heavily relies on the Protocol layer (access control) and the Tool/Resource implementation. A failure in access control (Protocol) effectively expands the attack surface for all other components.
• Methodological Advancement: By bridging the gap between static code analysis (CWE) and attack pattern modeling (CAPEC), the paper provides a robust framework for quantifying risk in AI agent ecosystems, moving beyond theoretical threat modeling to empirical code assessment.
• Community Guidance: The authors emphasize the need for standardized vulnerability reporting (e.g., SECURITY.md files) in open-source projects to facilitate ethical disclosure and remediation.

Limitations

• The study is limited to Python implementations; results may differ for TypeScript or other languages.
• Risk scores rely on MITRE metadata, which represents general patterns and may not perfectly reflect the specific risk context of every individual deployment.