Building a Robust Risk-Based Access Control System to Combat Ransomware's Capability to Encrypt

1. Problem Statement

Ransomware poses a critical threat by maliciously encrypting user data to demand a ransom. Traditional security measures often fail because:

• Static Policies: Conventional access control models (DAC, RBAC) lack the adaptability to detect dynamic, evolving threats in real-time.
• Detection Latency: Many existing solutions rely on post-hoc analysis, sandboxing, or hypervisor introspection, which introduce significant overhead or fail to block encryption before data loss occurs.
• Granularity: Approaches based solely on system calls (syscalls) or eBPF often lack the deep behavioral context needed to distinguish between legitimate cryptographic tools (e.g., OpenSSL) and ransomware, leading to false positives or missed detections.
• Enforcement Gap: Many detection systems alert administrators but lack a native, low-latency mechanism to actively block the malicious write operations immediately upon detection.

The authors propose a solution that combines probabilistic risk assessment with Mandatory Access Control (MAC) to block unauthorized encryption in real-time without disrupting legitimate workflows.

2. Methodology

The proposed system is a Dual-Layer Risk-Based Access Control (RiskBAC) architecture integrated into the Linux kernel via SELinux.

A. Data Collection & Feature Engineering

• Tracing Mechanism: Instead of relying on syscalls or eBPF, the system uses the native Linux ftrace framework with the function_graph tracer. This captures high-resolution kernel function execution traces, providing a hierarchical view of call relationships.
• Dataset Construction:
  • Sources: Benign applications, legitimate cryptographic tools (OpenSSL, GnuPG), and known Linux ransomware families.
  • Scope: Traces cover the first 1MB of data consumed by a process.
  • Features: The system extracts 113 engineered features, including:
    • Resource Counters: CPU%, Memory (RSS/VMS), I/O counts/bytes.
    • Graph Metrics: Betweenness centrality and clustering coefficients derived from the function call graph.
    • Function Frequencies: Counts of specific kernel function calls (e.g., fsnotify, locks_remove_file).
• Preprocessing: Includes outlier removal (PCA), handling missing values, and context-aware encoding to distinguish between similar function calls in different execution contexts.

B. Machine Learning & Rule Extraction

The system employs a two-layer detection strategy consuming the same feature vector:

1. Model Layer ($E_{ML}$): A XGBoost classifier trained to predict the probability of encryption ($P_{enc}$). It is optimized for high recall to catch evasive threats.
2. Rule Layer ($E_{RULE}$): A shallow decision tree extracted from the model (or trained independently) to provide interpretable, low-latency rules.
   • Feature Selection: Two pipelines were used. One identified a minimal 2-feature set (RSS, locks_remove_file) for extreme speed, but the authors expanded this to a 36-feature subset to improve robustness against throttled ransomware and benign high-I/O bursts.

C. Risk-Based Enforcement (SELinux Integration)

The core innovation is the integration of ML inference with SELinux via Common Intermediate Language (CIL):

• Risk Calculation: $R(x, ctx) = P_{enc}(x) \cdot I(ctx)$, where $I(ctx)$ weights the impact based on user, application, and file path.
• Enforcement Mechanism:
  • A user-space daemon monitors traces and computes verdicts.
  • If the risk exceeds a threshold ($\tau$), the daemon toggles two SELinux Booleans: rule_block and ml_block.
  • These booleans act as a "choke point" in the SELinux policy. If set to on, they deny write/append permissions to non-temporary user files for processes in the encryption_t domain.
• Whitelisting: The system supports granular whitelisting (User + App + Path). For example, user1 running openssl in $HOME/** is allowed, but the same binary running in /srv/shared or by user2 is blocked.

3. Key Contributions

1. Function-Level Tracing for Security: First work to systematically translate ftrace function graph data into ML-informed access control decisions, offering finer granularity than syscall-only telemetry.
2. Dual-Layer Architecture: A novel composition of a high-accuracy ML model and a fast, interpretable rule layer. This allows the system to block obvious attacks instantly (rules) while using the model for complex, borderline cases.
3. Native SELinux Integration: Unlike sandbox-based approaches, this system enforces blocks directly via the Linux Security Module (LSM) using CIL booleans, ensuring policy is auditable and enforceable at the kernel level without virtualization overhead.
4. Defender-Centric Utility: The system optimizes for a specific utility function ($U = 10 \cdot TP - 50 \cdot FN - 1 \cdot FP$), heavily penalizing False Negatives (missed ransomware) to prioritize data protection over minor false positives.
5. Comprehensive Dataset & Reproducibility: The authors constructed a specialized dataset of kernel traces and released the automation harness and raw data (excluding proprietary models) for reproducibility.

4. Experimental Results

The prototype was evaluated on a Linux workstation (Kernel 6.14) against benign workloads, crypto tools, and real-world ransomware samples.

• Detection Performance:
  • Model ($E_{ML}$): Achieved an F1 score of 0.97 and a Defender Utility of 1950.
  • Rules ($E_{RULE}$): Achieved an F1 of 0.95 with significantly lower latency.
  • Two-Layer OR: Combined, the system reached an F1 of 0.98 and Utility of 2050.
• Latency & Blocking:
  • Time-to-Block: The rule layer blocked attacks in ~10–19 ms (median), while the model took ~24–31 ms.
  • Data Protected: Attacks were stopped after writing only 10–20 KB of data, preventing significant data loss.
  • Zero-Day Capability: The system successfully detected ransomware families not present in the training set, proving it generalizes based on behavioral invariants rather than signature matching.
• Overhead:
  • The current user-space prototype showed high CPU spikes (up to 100% on P-cores) and RAM usage (~31–35 GB) during burst I/O due to unoptimized trace parsing.
  • However, the authors note this is an engineering limitation of the Python-based prototype; the enforcement hook (SELinux boolean toggle) itself is lightweight.
• Comparison:
  • Compared to eBPF (Brodzik et al.), the proposed system offers richer graph features and native SELinux policy integration, though eBPF has lower per-event latency (ns vs. ms).
  • Compared to Sandbox/VMI (Cuckoo), the proposed system has significantly lower operational overhead and faster end-to-end reaction times.

5. Significance and Future Work

• Significance: This research demonstrates that probabilistic risk assessment can be effectively coupled with deterministic MAC enforcement to create a resilient defense against ransomware. It bridges the gap between high-fidelity behavioral detection and immediate, auditable blocking.
• Future Directions:
  • Kernel Migration: Porting the trace collection and feature extraction from user-space to a kernel module (or eBPF) to eliminate the current CPU/RAM overhead and achieve sub-millisecond reaction times.
  • Adversarial Hardening: Defending against attackers attempting to manipulate timing or call patterns to evade detection.
  • Enterprise Automation: Automating the generation of per-tenant whitelists and expanding explainability for operational deployment.

In conclusion, the paper presents a practical, high-fidelity path to stopping ransomware encryption at the moment of initiation, leveraging the unique capabilities of Linux kernel tracing and SELinux to create a defense that is both explainable and highly effective.