Known Intents, New Combinations: Clause-Factorized Decoding for Compositional Multi-Intent Detection

This paper introduces the CoMIX-Shift benchmark and the ClauseCompose model to demonstrate that factorizing multi-intent detection into singleton intents significantly improves compositional generalization on unseen intent combinations and discourse patterns compared to traditional whole-utterance approaches.

The Gist

Imagine you are teaching a smart assistant (like Siri or Alexa) how to understand your voice commands.

The Old Way: The "Photo Album" Student

Most current AI models are trained like a student who memorizes a photo album.

• The Training: The student sees thousands of photos of specific combinations: "Play music AND turn on lights," or "Order pizza AND book a taxi."
• The Problem: If you ask the student to do something they've never seen in the album, like "Play music AND cancel my meeting," they get confused. Even though they know how to "play music" and how to "cancel a meeting" individually, they haven't memorized that specific photo of the two together.
• The Result: They fail because they are trying to recognize the whole picture rather than understanding the parts. They are great at recognizing familiar patterns but terrible at creativity.

The New Idea: The "Lego Builder"

This paper introduces a new way of thinking. Instead of memorizing whole pictures, the AI should learn to build with Lego bricks.

• The Training: The student learns what a "Music Brick" looks like and what a "Meeting Brick" looks like. They practice building single structures.
• The Magic: When you ask for a new combination ("Cancel meeting AND play music"), the AI doesn't panic. It simply grabs the "Cancel Meeting" brick and the "Play Music" brick and snaps them together. It doesn't need to have seen that exact combination before; it just needs to know how the individual bricks work.

The New Test: "CoMIX-Shift"

The authors realized that old tests were too easy. They were like a math test where the teacher only asked questions the students had already practiced.
To fix this, they built a new, harder test called CoMIX-Shift. It's like a "stress test" for the AI:

1. New Combinations: Give the AI two bricks it has never seen snapped together before.
2. New Words: Ask the same question but with different connecting words (e.g., "First do X, then do Y" vs. "Do X, after which do Y").
3. Messier Sentences: Add extra noise, like "Um, actually, please, could you..." to see if the AI gets distracted.

The Results: The "ClauseCompose" Model

The paper tested a new, lightweight model called ClauseCompose (the Lego Builder) against the old models (the Photo Album students).

• On Easy Tests (Familiar Combinations): The old models did fine. They just recognized the photo they memorized.
• On Hard Tests (New Combinations): The old models crashed. Their accuracy dropped to near zero because they saw a "new photo" they hadn't memorized.
• The Winner: ClauseCompose soared. Because it understood the individual "bricks" (clauses), it could handle:
  • New pairs of intents (95.7% success).
  • Messy, long sentences (62.5% success).
  • Completely new sentence structures (91.1% success).

The Big Takeaway

The paper argues that we are testing AI the wrong way. We keep asking, "Can you recognize this specific pattern?" when we should be asking, "Can you combine what you know to solve a new problem?"

In simple terms:
If you want a smart assistant that can handle real life, don't just teach it to memorize every possible sentence a human might say. Teach it to understand the building blocks of language so it can build new sentences on the fly. A simple, structured approach (like Lego) actually beats a complex, "big brain" approach (like a photo album) when it comes to handling the unexpected.

Technical Summary

1. Problem Statement

Current Multi-Intent Detection (MID) research primarily focuses on whether a model can identify multiple intents within a single utterance. However, existing benchmarks and evaluation methods are flawed because they typically split data such that training and testing sets share the same co-occurrence patterns of intent combinations.

• The Core Issue: Models often succeed by memorizing familiar intent pairings (surface statistics) rather than learning true compositional generalization.
• Real-World Failure Mode: Deployed assistants fail when users combine known capabilities in unfamiliar ways (e.g., "cancel my meeting with Sam, and then play Phoebe Bridgers") if that specific pair was never seen during training.
• The Gap: There is a lack of rigorous evaluation that tests a model's ability to generalize to new combinations of familiar intents, varying discourse patterns, and unseen structural templates.

2. Methodology

A. The Benchmark: CoMIX-Shift

The authors introduce CoMIX-Shift, a controlled synthetic benchmark designed to stress-test compositional generalization. It is built from 10 atomic intents (e.g., check_weather, play_music, cancel_meeting) and includes:

• Data Generation: Singleton utterances are created using templates with slot values. Multi-intent utterances are formed by composing these clauses with various discourse connectors (e.g., "and," "then," "before anything else").
• Split Design:
  • Training: Uses only singleton examples and "seen" intent pairs. No triples are included in training.
  • Testing: Includes six distinct settings to isolate different failure modes:
    1. Seen Pairs: Familiar combinations.
    2. Unseen Pairs: Held-out intent pairs (never seen together in training).
    3. Pair Shift: Unseen pairs with new discourse connectors.
    4. Long Shift: Unseen pairs wrapped in longer, noisier scaffolds.
    5. Template Holdout: Unseen pairs using clause templates never seen in training.
    6. Triple Shift: Zero-shot triples (3 intents) with shifted discourse.

B. The Proposed Model: ClauseCompose

The authors propose ClauseCompose, a lightweight, clause-factorized decoder.

• Training: Trained only on singleton intents (one intent per utterance). It does not see multi-intent examples during training.
• Inference:
  1. Segmentation: The input utterance is segmented into clauses using a small, rule-based discourse-marker grammar.
  2. Classification: Each segment is encoded independently by the singleton-trained encoder.
  3. Decoding: A greedy decoder selects the highest-scoring intent for each segment, preventing duplicates to form the final intent set.
• Philosophy: This approach hard-codes a structural prior: multi-intent utterances are compositions of single-intent clauses. This allows the model to handle unseen arities (e.g., triples) and new combinations without retraining.

C. Baselines

The study compares ClauseCompose against:

1. AtomicSet: Treats every seen intent set as a single class (brittle baseline).
2. WholeMultiLabel: A whole-utterance multi-label model with a learned count head (predicts 1, 2, or 3 intents).
3. BERTTinyMultiLabel: A fine-tuned tiny BERT encoder with the same multi-label/count objective.
4. ClauseComposeOracle: An oracle version of ClauseCompose that uses gold-standard clause boundaries to isolate segmentation errors.

3. Key Contributions

1. CoMIX-Shift Benchmark: A controlled benchmark featuring held-out pairs, discourse shifts, longer/noisier wrappers, held-out templates, and zero-shot triples to explicitly measure compositional generalization.
2. ClauseCompose: A novel, lightweight architecture that trains exclusively on singletons and factorizes multi-intent decoding, proving that simple structural priors can outperform complex whole-utterance models in compositional settings.
3. Enhanced Evaluation: The inclusion of a fine-tuned BERT baseline and a manually authored, SNIPS-style compositional set (240 examples) to validate findings beyond synthetic data.
4. Evaluation Paradigm Shift: Demonstrates that model rankings flip when evaluation moves from standard splits to compositional splits; whole-utterance models excel at memorization but fail at generalization, while factorized models dominate generalization.

4. Results

The results highlight a stark contrast between "in-distribution" performance and "compositional" performance.

Setting	WholeMultiLabel (EM)	BERTTiny (EM)	ClauseCompose (EM)
Seen Pairs	100.0	100.0	95.2
Unseen Pairs	81.4	91.5	95.7
Pair Shift (New Connectors)	55.7	77.6	93.9
Long Shift (Noisy/Long)	18.8	48.9	62.5
Template Holdout	15.5	11.0	49.8
Triple Shift (Zero-shot)	0.0	0.0	91.1

• Key Findings:
  • Zero-Shot Triples: Whole-utterance models (including BERT) collapsed to 0.0% exact match on unseen triples because their count heads were never trained to emit three intents. ClauseCompose achieved 91.1%.
  • Discourse Shift: On unseen pairs with new connectors, ClauseCompose (93.9%) significantly outperformed the BERT baseline (77.6%) and WholeMultiLabel (55.7%).
  • Manual SNIPS Check: On a small, manually authored dataset with held-out pairs, ClauseCompose achieved 97.5% exact match, while WholeMultiLabel dropped to 41.3%.
  • Robustness: Even with noisy wrappers and longer sentences, ClauseCompose maintained a lead of 13.6 points over the strongest whole-utterance baseline.

5. Significance and Conclusion

• Re-evaluating MID: The paper argues that standard MID benchmarks are too easy and encourage memorization of co-occurrence patterns rather than true understanding of intent composition.
• Value of Simplicity: The study demonstrates that explicit factorization (treating multi-intent utterances as compositions of single clauses) is a far more effective strategy for compositional generalization than complex, whole-utterance pattern matching, even when using lightweight models.
• Deployment Implications: For real-world assistants, the ability to recombine known intents in novel ways is critical. The results suggest that future MID systems should prioritize compositional evaluation and potentially adopt factorized architectures to handle the "long tail" of user requests.
• Limitations: The authors acknowledge that the benchmark is synthetic and templatic. While it isolates composition cleanly, real-world language is messier. Future work needs to combine these findings with real mixed-intent data and explicit holdouts for slots and templates.

In summary, the paper establishes that compositional generalization is the missing axis in Multi-Intent Detection evaluation and proves that clause-factorized decoding is a robust solution to this problem, outperforming state-of-the-art whole-utterance models when faced with unseen intent combinations.