WSymBert: A Weighted Neuro-Symbolic Attention Model for Interpretable CVSS Prediction

Technical Summary: WSymBert – A Weighted Neuro-Symbolic Attention Model for Interpretable CVSS Prediction

Problem Definition
The rapid proliferation of software vulnerabilities has created a critical bottleneck in security operations: the manual assessment of Common Vulnerability Scoring System (CVSS) metrics is slow, expert-dependent, and difficult to scale. Consequently, many vulnerabilities remain unscored for extended periods, hindering effective patch prioritization and risk mitigation. While automated approaches using transformer-based models (e.g., BERT, SBERT) have improved prediction accuracy, they often function as "black boxes." These models lack transparency, relying on statistical patterns rather than domain logic, and their explanations (often generated via post-hoc methods) may highlight irrelevant tokens or fail to align with expert reasoning. The challenge is to develop a system that not only predicts the eight CVSS v3.1 Base metrics (Attack Vector, Attack Complexity, Privileges Required, User Interaction, Scope, Confidentiality, Integrity, Availability) with high accuracy but also provides interpretable reasoning consistent with cybersecurity domain knowledge.

Methodology: WSymBert Architecture
The authors propose WSymBert, a weighted neuro-symbolic framework that integrates contextual language understanding with structured expert knowledge. The architecture consists of three primary components:

1. Semantic Encoding: The model utilizes a fine-tuned SBERT encoder to generate contextual token embeddings ($H$) from unstructured vulnerability descriptions.
2. Symbolic Knowledge Extraction: Expert-defined decision trees, derived from official CVSS guidelines and security ontologies, are used to construct "expert lexicons." These lexicons map specific descriptive phrases (e.g., "requires local access") to CVSS metric values. Additionally, KeyBERT is used to extract model-relevant keywords from the input text.
3. Weighted Symbolic Attention: This is the core innovation. Instead of relying solely on learned attention, WSymBert introduces a symbolic weighting mechanism.
   • For each token, a symbolic weight ($w_i$) is assigned based on its presence in the intersection of the expert lexicon and the model's extracted keywords.
   • Tokens confirmed by both expert logic and model semantics receive a high weight (10), those in the symmetric difference receive a weight of 1, and others receive 0.
   • To prevent "silent failure" where a lack of symbolic matches collapses the vector, a baseline weight of 1 is added, ensuring the model can fall back to learned pooling ($\alpha_i$) when symbolic cues are absent.
   • The final pooled representation ($z_m$) for each of the eight metrics is computed by fusing the learned attention scores with these symbolic weights.
4. Multi-Head Prediction: Eight independent linear classifiers are trained on the fused representations to predict the categorical values for each Base metric. The SBERT backbone is shared across all heads to minimize training costs and act as a regularizer.

Key Contributions

• Neuro-Symbolic Framework: The paper introduces WSymBert, a hybrid model that predicts all eight CVSS v3.1 Base metrics by explicitly integrating symbolic expert knowledge into the attention mechanism of a transformer model.
• Guided Attention: Unlike post-hoc explainability methods, the model incorporates structured guidance directly into the decision process, steering attention toward security-relevant cues defined by expert rules.
• Robust Performance: The model is evaluated on two benchmark datasets (the "Combine" dataset and a curated "NVD" dataset), demonstrating consistent outperformance over strong baselines (SBERT and DistilBERT), particularly on imbalanced and difficult metrics.
• Quantitative Explainability: The study provides a rigorous analysis showing that WSymBert's predictions rely more heavily on domain-specific evidence and less on irrelevant tokens (stopwords, numbers) compared to baseline models.

Experimental Results
The model was evaluated on the Combine and NVD datasets using Accuracy, Precision, Recall, and F1-score.

• Accuracy: WSymBert achieved an average accuracy of 96.2% on the NVD dataset and approximately 95% on the Combine dataset, outperforming SBERT (92% and 92% respectively) and DistilBERT (89% and 89% respectively).
• Imbalanced Metrics: Significant improvements were observed in metrics with skewed class distributions, such as Attack Complexity (AC) and Privileges Required (PR). For instance, on the Combine dataset, WSymBert achieved an F1-score of 97.76% for AC, compared to 56.10% for DistilBERT and 83.70% for SBERT.
• Error Reduction: WSymBert demonstrated lower Mean Squared Error (MSE) and Mean Absolute Error (MAE) in predicting the final numerical CVSS score compared to baselines.
• Interpretability Analysis: Using Integrated Gradients (IG), the study found that WSymBert's top-10 influential tokens had a "Noise Rate" (proportion of stopwords, numbers, or short tokens) of less than 5%, whereas DistilBERT exceeded 30%. Furthermore, WSymBert exhibited lower Infidelity and higher Completeness in its explanations, indicating that its attributions are more faithful to the model's actual reasoning process.

Significance and Claims
The paper claims that WSymBert offers a practical solution for automated vulnerability severity assessment by bridging the gap between high-accuracy neural prediction and domain-aligned interpretability. By embedding symbolic constraints directly into the attention layer, the model ensures that its reasoning is traceable to meaningful evidence in the text, addressing the "black box" limitation of standard transformers. The authors assert that this approach provides a trustworthy foundation for security analysts, enabling them to justify severity decisions with explanations that align with expert logic rather than statistical shortcuts. The work demonstrates that combining neural learning with symbolic guidance is a viable path toward responsible and explainable AI in cybersecurity.

Limitations and Future Directions
The authors acknowledge that reliance on predefined symbolic cues could introduce a potential attack surface where adversaries might manipulate vulnerability descriptions with salient lexicon cues to inflate scores. Future work suggests integrating verification mechanisms and confidence calibration to mitigate this. Additionally, the authors propose extending the model to incorporate external structured data (CWE, CPE), dynamic environmental metrics, and knowledge graphs to further enhance context understanding and real-time applicability.