Technical Summary: SFL-MTSC for Robust Multi-Intent Spoken Language Understanding

1. Problem Statement

Prompt-based Spoken Language Understanding (SLU) utilizing Large Language Models (LLMs) faces significant challenges in multi-intent scenarios. Due to the inherent stochasticity of LLM decoding, different reasoning paths often produce inconsistent intent–slot structures. While Self-Consistency methods (sampling multiple paths and aggregating via majority voting) have improved reasoning in other domains, standard output-level majority voting is insufficient for SLU. In multi-intent settings, conflicting semantic frames across different decoding runs make it difficult to aggregate predictions at the output level without losing valid information or propagating hallucinations. Existing SLU self-consistency approaches are not designed for the structured, frame-based predictions required in multi-intent domains, and alternative methods like LLM-as-a-judge introduce additional latency and hallucination risks while lacking fine-grained filtering capabilities.

2. Methodology: SFL-MTSC

The authors propose Semantic Frame-Level Multi-Task Self-Consistency (SFL-MTSC), a structured aggregation framework that operates below the final output level. Instead of voting on complete outputs, SFL-MTSC decomposes predictions into semantic frames, clusters them, and re-integrates reliable components. The workflow consists of the following stages:

2.1 Multi-Path Reasoning and Frame Pool Construction

Given an input utterance $X$, the system performs inference $K$ times using distinct sampling temperatures to generate $K$ reasoning paths. Each path produces a set of semantic frames $F = (d, i, s)$, where $d$ is the domain, $i$ is the intent, and $s$ is a set of slot–value pairs. These frames are collected into a unified Frame Pool, discarding any frames with missing domain or intent fields.

2.2 Frame-Level Clustering

The framework employs a coarse-to-fine clustering procedure to group frames referring to the same underlying semantic intent:

1. Domain–Intent Clustering: Frames are first partitioned into buckets based on their $(domain, intent)$ pairs. This prevents merging frames with different intents and narrows the search space for slot comparison.
2. Slot Clustering: Within each bucket, frames are clustered based on slot similarity. The authors introduce Hybrid Jaccard similarity, which interpolates between:
   • Key–Value Jaccard ($sim_{kv}$): Structurally precise but sensitive to variations in slot key naming across paths.
   • Value-Based Jaccard ($sim_{val}$): More robust to key variations but prone to conflating frames with coincidentally overlapping values.
   • A mixing coefficient $\alpha$ balances these two metrics. Connected components in a threshold similarity graph form the final slot clusters.

2.3 Path Support Scoring and Filtering

Inspired by association rule mining, the framework evaluates the reliability of each cluster by calculating its path support score ($supp(C)$), defined as the number of distinct reasoning paths that contributed frames to that cluster.

• Filtering: Clusters supported by fewer than half of the total paths ($K/2$) are discarded. This step filters out hallucinated intents and structurally inconsistent slot predictions that lack cross-path agreement.

2.4 Re-Integration

The retained clusters are re-integrated to form the final prediction $Y^*$:

• Domain and Intent: Directly adopted from the bucket key.
• Slots: A Value-First strategy is employed. Representative slot values are identified based on their support count (retaining values supported by at least half the frames in the cluster). Corresponding slot keys are then determined via majority vote for each retained value.

3. Key Contributions

The paper outlines three primary contributions:

1. SFL-MTSC Framework: A novel frame-level self-consistency mechanism for multi-intent SLU. By operating below the output level, it effectively removes false intents and noisy slot predictions that standard majority voting cannot resolve.
2. Hybrid Jaccard Similarity: A new metric for slot clustering that combines key-based and value-based matching, enhancing robustness when slot key names vary across different reasoning paths.
3. Comprehensive Zero-Shot Evaluation: Extensive experiments on the MAC-SLU benchmark (a Chinese multi-intent dataset) across three distinct model configurations: text-only LLMs, ASR+LLM pipelines, and end-to-end Large Audio-Language Models (LALMs).

4. Experimental Results

Experiments were conducted in a zero-shot setting on the MAC-SLU dataset using $K=5$ reasoning paths. The baseline methods included Vanilla Prompting, CroPrompt, and GPT-SLU.

• Overall Performance: SFL-MTSC consistently improved Overall Accuracy and Slot F1 across all model configurations and prompting baselines.
• Impact on Baselines: The most significant gains were observed with Vanilla Prompting, where Slot F1 increased by up to +28.86% and Overall Accuracy by +1.45% (depending on the model). This suggests that frame-level aggregation is particularly effective when the base prompting strategy lacks structural guidance.
• Intent vs. Slot Stability: While Slot F1 showed consistent improvements, Intent Accuracy remained largely stable or experienced minor fluctuations (sometimes slight decreases). This indicates that cross-path inconsistency in zero-shot inference manifests primarily at the slot level, while intent predictions are relatively stable even without aggregation.
• Ablation Studies:
  • Filtering: Slot-level filtering was identified as the primary driver of performance gains. Adding intent-level filtering provided only marginal improvements.
  • Hybrid Jaccard Coefficient ($\alpha$): Overall Accuracy was largely insensitive to $\alpha$ in the range $[0.0, 0.7]$. Slot F1 peaked at $\alpha=0.1$, suggesting value-based similarity is slightly more critical, though $\alpha=0.3$ was selected as the default to balance metrics.

5. Significance and Claims

The authors claim that SFL-MTSC offers a robust, post-inference strategy for multi-intent SLU that requires no additional supervision or fine-tuning. By decomposing predictions into semantic frames and applying path support scoring, the framework effectively resolves the structural inconsistencies inherent in stochastic LLM decoding.

The paper notes specific limitations:

• Occasional drops in Intent Accuracy.
• Diminished gains in LALM settings, likely due to high decoding variance in end-to-end models.
• Evaluation limited to a single dataset (MAC-SLU).

Future work is proposed to explore finer-grained intent clustering and aggregation mechanisms better suited for LALMs. The code and data are made publicly available to support further research in this area.