Technical Summary: DGVoiC – Speaker Clustering for Fraud Investigation

1. Problem Statement

Insurance fraud remains a significant financial burden, with estimated costs reaching £14.4 billion globally in 2023–2024. A critical challenge in fraud detection lies within call-centre workflows, particularly at the First Notice of Loss (FNOL) stage. While current fraud detection systems predominantly rely on structured data, text transcripts, or images, they largely underutilize repeated speaker identity across calls as an investigative signal.

Existing research on fraudulent phone calls focuses heavily on transcript content or scam classification rather than persistent speaker identity. Furthermore, real-world operational call-centre audio is difficult to study due to privacy and biometric data constraints, creating a gap between advances in speaker representation learning and the practical needs of fraud investigation. Investigators often struggle to verify if a customer is the actual speaker or to identify if the same voice appears across multiple, distinct customer profiles, a pattern that may indicate organized misuse.

2. Methodology: The DGVoiC Framework

The paper proposes DGVoiC, a voice clustering framework designed to operate on anonymized, real-world call-centre audio. The system is not intended as a standalone automated fraud decision engine but as a support tool to assist analysts in verifying speaker consistency and surfacing cross-profile voice links.

A. Data Preprocessing and Anonymization

Given the use of real operational data, the pipeline prioritizes privacy:

• Anonymization: Personally Identifiable Information (PII) and sensitive attributes are detected using Named Entity Recognition (NER), Regex, and word-level timestamps from WhisperX.
• Audio Muting: Corresponding audio regions containing sensitive data are muted using librosa and soundfile before analysis.
• Preprocessing: Long non-speech regions are removed using Resemblyzer's preprocess_wav to reduce the impact of silence and focus the representation on active speech.

B. Speaker Embedding Extraction

The framework employs a sliding-window strategy to handle long and variable telephony calls:

• Segmentation: An overlapping sliding-window approach is used (tested with 6s windows and 3s hop sizes) to ensure short, informative speech regions are not missed.
• Embedding Model: ECAPA-TDNN is used to encode valid segments into fixed-dimensional speaker embeddings.
• Aggregation: Segment-level embeddings for a single customer interaction are aggregated via mean pooling to form a final voice representation.

C. Clustering and Risk Scoring

• Similarity Metric: Cosine similarity is used to compare embeddings across different claims and customer profiles.
• Clustering Logic: The system identifies repeated speakers even when customer details differ.
• Investigation Triggers: To support analyst triage, the system flags cases meeting specific conditions:
  1. Cosine similarity > 0.718 between candidate voice representations.
  2. Membership in a cluster containing more than 4 distinct customer profiles.
  3. Consistency of the cluster across a customer's recordings.
• Risk Scoring: A bounded probability score is generated using a sigmoid function to prioritize analyst review. Cases with high similarity and large cluster sizes (e.g., ≥9 profiles) receive higher risk scores.

3. Experimental Setup

• Dataset: 121 real call recordings from multiple customer profiles.
• Selection Criteria: Restricted to customers with at least four recordings to assess within-customer variation.
• Ground Truth: A curated subset of 56 samples (organized into 22 human-agreed clusters) was created by two independent fraud-domain practitioners from Domestic & General. Only samples where both reviewers achieved high-confidence agreement were retained.
• Hardware: Experiments were run on Databricks custom compute (32 CPU cores, 128 GB RAM, single 64 GB GPU).

4. Key Results

The proposed configuration (Resemblyzer preprocessing + ECAPA-TDNN + 6s sliding window + mean pooling + 0.718 threshold) achieved the following performance metrics on the validation subset:

• Adjusted Mutual Information (AMI): 96%
• Adjusted Rand Index (ARI): 95%
• Completeness: 98%
• Homogeneity: 100%
• V-measure: 99%
• Accuracy: 95%
• F1 Score: 0.96

Latency:

• Embedding extraction: ~10 seconds per recording.
• Clustering and linking: ~84ms per recording.
• Total End-to-End Processing: ~10.08 seconds per recording.

Auxiliary Verification Metrics:
At the 0.718 clustering threshold, the system yielded an Equal Error Rate (EER) of 3.85%, a False Acceptance Rate (FAR) of 0.50%, and a False Rejection Rate (FRR) of 9.62%.

Error Analysis:
The model produced 24 clusters compared to the 22 ground-truth clusters. The discrepancies were attributed to residual acoustic variability, such as a customer sounding noticeably different in two calls or sustained loud background noise shifting a recording's representation.

5. Significance and Contributions

The paper claims the following contributions and significance:

1. Practical Framework: It introduces DGVoiC, a practical voice clustering approach specifically designed for anonymized real insurance call-centre data, addressing a gap where most research relies on synthetic or benchmark datasets.
2. Pipeline Innovation: It proposes a pipeline tailored for long, variable telephony calls, combining word-level aligned anonymization, speech-focused preprocessing, and sliding-window embedding extraction.
3. Operational Utility: The results demonstrate that ECAPA-based speaker embeddings can achieve strong agreement with human-reviewed clusters under real conditions. This provides a "strong additional signal" for fraud investigation, helping analysts verify speaker consistency and identify repeated voices across different customer profiles.
4. Contextual Positioning: Unlike prior work focused on caller authentication (verifying known users) or transcript analysis, DGVoiC targets customer-level speaker clustering to detect cross-profile voice reuse. The authors note that direct numerical comparison with state-of-the-art diarization systems is methodologically inappropriate due to differing tasks and metrics; instead, they position DGVoiC as a lower-complexity, efficient solution designed to support analyst-led investigation rather than automated decision-making.

The paper concludes that while the system is effective, future work will focus on improving robustness against challenging conditions (background noise, within-speaker variability) and exploring complementary audio cues like prosodic and behavioral features.