Technical Summary: MCA2 – Multi-View Text Anomaly Detection with Contrastive Collaboration and Adaptive Allocation

1. Problem Definition

Text Anomaly Detection (TAD) aims to identify textual instances that deviate from normal patterns, a critical task for applications such as harmful content moderation, phishing detection, and spam filtering. While current state-of-the-art methods typically follow a two-step "embedding–detector" pipeline—using a single pretrained language model (e.g., BERT, OpenAI embeddings) to generate vector representations followed by an anomaly detector (e.g., LOF, iForest)—they suffer from inherent limitations:

• Single-Perspective Bias: A single embedding model is biased toward its specific pretraining corpus distribution, failing to provide robust representations across diverse domains, anomaly types, and textual styles.
• Detector Incompatibility: Different embedding models produce distinct distributional characteristics, making it difficult to pre-select a compatible anomaly detector for a given dataset without costly exhaustive evaluation.
• Lack of Adaptability: Existing methods do not dynamically adjust to the varying utility of different embeddings across different datasets or even different samples within a dataset.

The paper formulates TAD as a Multi-View Anomaly Detection problem, where embeddings from $K$ different large language models constitute $K$ complementary views of the same text data. The goal is to learn an anomaly scoring function that leverages these multiple views to outperform single-view baselines.

2. Methodology: The MCA2 Framework

The authors propose MCA2 (Multi-view TAD framework with Contrastive Collaboration and Adaptive Allocation), which operates under a one-class anomaly detection setting (training only on normal data). The framework consists of three core components:

2.1 Multi-View Reconstruction Backbone

To capture intra-view patterns, MCA2 employs an independent autoencoder for each embedding view $k$.

• Architecture: For each view $k$, an MLP-based autoencoder maps the input embedding $v^{(k)}_i$ to a latent representation $z^{(k)}_i$ and reconstructs it as $\hat{v}^{(k)}_i$.
• Objective: The model minimizes the squared $\ell_2$ reconstruction error.
• Scoring: The reconstruction error serves as the intra-view anomaly score ($s^{(k)}_{i,rec}$), where higher errors indicate greater deviation from the learned normal distribution of that specific view.

2.2 Inter-View Contrastive Collaboration

To exploit the complementary nature of different views and enforce consistency, a contrastive collaboration module is introduced.

• Mechanism: The module maximizes the mutual information between the latent representations of the same sample across different views ($j$ and $k$) using an InfoNCE contrastive loss.
• Negative Sampling: It treats samples from other views and other instances within the same view as negative pairs to learn discriminative representations.
• Scoring: Normal samples are expected to exhibit strong cross-view correspondence. Anomalous samples may break this consistency. The inter-view anomaly score ($s^{(k)}_{i,con}$) is derived from the negative log-probability of matching across views, serving as a complementary signal to the reconstruction error.

2.3 Adaptive View Contribution Allocation

Recognizing that different views contribute unequally to anomaly detection depending on the dataset and the specific sample, MCA2 includes an adaptive allocation module.

• Alignment: Principal Component Analysis (PCA) is first applied to each view to align dimensions and retain principal structural information.
• Weight Estimation: A lightweight MLP-based estimator takes the aligned features of a sample and outputs contribution weights ($w^{(k)}_i$) for each view. This allows for sample-level adaptivity, where the model can dynamically emphasize the most informative view for a specific instance.
• Final Scoring: The final anomaly score is a weighted sum of the reconstruction and contrastive scores across all views:
  $$s(x_i) = \sum_{k=1}^{K} w^{(k)}_i (\alpha s^{(k)}_{i,rec} + \beta s^{(k)}_{i,con})$$
• Training Strategy: To ensure stability, MCA2 uses a two-stage training approach:
  1. Stage 1: The allocation module is frozen (uniform weights) to train the encoders/decoders and contrastive mechanisms.
  2. Stage 2: The detection backbone is frozen, and only the allocation module is trained to learn optimal view weights based on the reliable scores from Stage 1.

3. Key Contributions

The paper outlines three primary contributions:

1. New Paradigm: It is the first work to leverage embeddings from multiple pretrained language models simultaneously to capture complementary information for TAD, moving beyond the single-model constraint.
2. Novel Method (MCA2): A unified framework that integrates a multi-view reconstruction backbone with a contrastive collaboration module (to align views) and an adaptive allocation module (to weight views dynamically).
3. Extensive Empirical Validation: Comprehensive experiments on 10 real-world benchmark datasets demonstrating superior performance over strong baselines.

4. Experimental Results

The authors evaluated MCA2 on 10 datasets from NLP-ADBench and TAD-Bench, comparing it against end-to-end methods, single-view embedding-based methods, and existing multi-view baselines (NCMOD, RCPMOD).

• Performance: MCA2 achieved the best AUROC on 9 out of 10 datasets. It consistently outperformed single-view baselines (e.g., BERT+LOF, OpenAI+LUNAR) and multi-view baselines.
• Ablation Studies: Removing any component (Adaptive Allocation, Contrastive Collaboration, or Autoencoder) led to performance degradation. The Contrastive Collaboration module was found to be the most critical, with its removal causing the largest drops (e.g., -17.89% to -23.32% on specific datasets), highlighting the importance of inter-view alignment.
• Robustness: MCA2 demonstrated strong robustness against contaminated training data (inlier noise), maintaining stable performance even as the ratio of injected anomalies in the training set increased, outperforming strong baselines like OpenAI-L+LUNAR.
• Adaptivity: Visualization of the allocation module showed that the model does not uniformly select a fixed view but dynamically shifts preferences based on the dataset (e.g., favoring OpenAI-ada for BBCNews vs. OpenAI-large for AGNews) and even at the sample level.

5. Significance and Claims

The paper claims that MCA2 addresses the critical limitation of single-perspective TAD by effectively harnessing the diversity of modern language models. By treating embeddings from different models as complementary views and using contrastive learning to align them while adaptively weighting their contributions, MCA2 achieves a more robust and generalizable anomaly detection system.

The authors note that while MCA2 achieves state-of-the-art results, it currently incurs higher inference costs due to the reliance on multiple external embedding models. They suggest that future work could focus on more efficient strategies, such as dynamically selecting a subset of views or incrementally requesting embeddings, to improve scalability in resource-constrained environments. The research is presented with ethical considerations, noting the use of public datasets and the importance of managing false alarms in real-world deployment to prevent bias.