Syn-TurnTurk: A Synthetic Dataset for Turn-Taking Prediction in Turkish Dialogues

1. Problem Statement

The paper addresses a critical bottleneck in voice-based conversational AI: natural turn-taking prediction.

• Current Limitations: Most voice chatbots rely on simple silence detection (e.g., waiting for a fixed duration of silence) to determine when a user has finished speaking. This approach is unreliable because human speech involves irregular pauses, mid-sentence breaks, and strategic silences. Consequently, bots frequently interrupt users, disrupting conversational flow.
• The Language Gap: While solutions exist for English, there is a severe lack of high-quality, labeled datasets for Turkish. Turkish has a unique suffix-based grammar and sentence structure that causes models trained on English data to fail in capturing the specific linguistic cues required to predict turn boundaries accurately.
• Benchmark Deficit: The field lacks established benchmarks and ground truth data to compare turn-taking models effectively, particularly for non-English languages.

2. Methodology

The authors propose a two-pronged methodology: the creation of a synthetic dataset and the evaluation of various predictive models.

A. Dataset Generation (Syn-TurnTurk)

Instead of relying on scarce real-world recordings, the authors generated a synthetic dataset using five distinct Qwen Large Language Models (LLMs) via API calls:

• Models Used: qwen3-max, qwen3.5-35b, qwen3.5-plus, qwen3.5-397b, and qwen3.5-flash.
• Generation Strategy:
  • Topics: 79 unique conversational topics were randomly assigned to ensure thematic diversity.
  • Human-like Features: Models were explicitly instructed to generate dialogues containing overlaps, strategic silences, and interjections to mimic real human interaction.
  • Variability: The temperature parameter was adjusted (mostly set to 0.7) to balance coherence with conversational spontaneity.
• Dataset Statistics:
  • Total Dialogues: 1,625.
  • Speaker Changes: 12,560.
  • Overlaps: 5,305 documented instances (2,213.5 seconds of concurrent speech).
  • Temporal Metrics: The dataset exhibits an asymmetric distribution of Floor Transfer Offsets (FTO), with a mean of 0.286s and a median of 0.743s, reflecting both rapid transitions and longer pauses.

B. Modeling and Evaluation

The authors evaluated the dataset's utility by training and testing several classification architectures to distinguish between "intentional pauses" (turn continues) and "completed turns" (turn ends).

• Data Labeling:
  • Positive Class (1): The final one-third of a text sequence (indicating a turn end).
  • Negative Class (0): Two random segments from the remaining text (indicating a turn continues).
  • Class Balance: The dataset was downsampled to 1,306 positive and 2,696 negative samples for training to ensure comparability.
• Feature Representation: The intfloat/multilingual-e5-large embedding model was used to capture Turkish semantic and structural nuances.
• Algorithms Tested:
  • Traditional ML: Logistic Regression (LR), Decision Tree (DT), Random Forest (RF).
  • Deep Learning: LSTM, Bidirectional LSTM (BI-LSTM).
  • Ensemble: A soft-voting combination of LR and RF.
• Validation: A 5-fold cross-validation approach was employed.

3. Key Contributions

1. Syn-TurnTurk Dataset: The first synthetic Turkish dialogue dataset specifically designed for turn-taking prediction, featuring realistic overlaps and silence patterns generated by state-of-the-art LLMs.
2. Linguistic Adaptation: A shift from time-based silence detection to linguistic structure analysis, leveraging Turkish grammatical markers to predict turn boundaries.
3. Benchmarking: The paper provides a comprehensive performance baseline across 30 model combinations, establishing a reference point for future research in Turkish conversational AI.
4. Open Access: The dataset is hosted on Hugging Face, promoting reproducibility and further development in the Turkish NLP community.

4. Results

The experimental results, summarized in Table III of the paper, highlight the following:

• Top Performers:
  • BI-LSTM: Achieved the highest Accuracy (0.839) and a strong AUC (0.907).
  • Ensemble (LR+RF): Achieved the highest AUC (0.910) and a competitive Accuracy (0.836).
• Model Comparison:
  • Deep learning models (LSTM/BI-LSTM) and the Ensemble method significantly outperformed traditional Decision Trees and standalone Logistic Regression.
  • The BI-LSTM architecture proved particularly effective, confirming that understanding the bidirectional linguistic flow is crucial for Turkish turn-taking.
• Model Complexity Insight: The subset generated by the most advanced model (qwen3.5-397b-a17b) showed slightly lower prediction performance. The authors attribute this to the increased linguistic complexity and more natural, fluid conversational patterns of this model, which makes turn boundaries harder for classifiers to detect compared to simpler outputs.

5. Significance

• Advancing Turkish AI: This work fills a critical resource gap for Turkish NLP, enabling the development of voice assistants that feel more "human" and less mechanical.
• Beyond Silence Detection: It demonstrates that relying solely on acoustic silence is insufficient; successful turn-taking requires modeling linguistic cues and context.
• Scalability: The synthetic generation approach offers a scalable solution for creating high-quality training data for other under-resourced languages where real-world annotated data is scarce.
• Practical Application: The high accuracy and AUC scores suggest that these models can be deployed to reduce interruptions in real-world voice chatbots, significantly improving user experience and conversational flow.