DEBISS: a Corpus of Individual, Semi-structured and Spoken Debates

This paper introduces DEBISS, a novel corpus of spoken and individual semi-structured debates designed to address the scarcity of debate datasets by providing comprehensive NLP annotations for tasks such as speech-to-text, speaker diarization, argument mining, and debater quality assessment.

The Gist

Imagine you want to teach a robot how to argue like a human. You can't just feed it a dictionary; you need to show it real conversations where people think on their feet, interrupt each other, stumble over words, and try to convince one another.

For a long time, scientists trying to build these "argument robots" have been stuck with a limited menu. They mostly had two options:

1. The "Political Theater" Menu: Highly scripted, formal debates between politicians (like presidential debates). These are great, but they are too polished and rigid. Real people don't talk like that.
2. The "Internet Comment Section" Menu: Written posts on Twitter or Reddit. These are messy and real, but they lack the sound of a human voice, the pauses, the "umms," and the energy of a face-to-face conversation.

Enter DEBISS: The "Real-Life Debate" Recipe Book.

This paper introduces a new, special collection of data called DEBISS (Spoken, Individual, and Semi-structured Debates). Think of it as a high-definition recording studio where 67 computer science students from a Brazilian university got together to debate a hot topic: "Generative Artificial Intelligence and its impact on society."

Here is what makes DEBISS special, explained through some everyday analogies:

1. The "Semi-Structured" Sandwich

Most debates are either a rigid script (like a play) or a chaotic free-for-all (like a mosh pit). DEBISS is the perfect sandwich.

• The Bread (Structure): There are rules. A moderator asks specific questions, and students have to answer them. This keeps the conversation on track.
• The Filling (Spontaneity): Between the questions, students can speak freely, interrupt (politely), and react to each other. This captures the messy, human "flow" of real conversation that computers usually miss.

2. The "Individual" Spotlight

In many group debates, people speak as a team, blending their voices into one "we." In DEBISS, every student is an individual soloist. Even though they are in a group of 3 to 5, each person defends their own unique viewpoint. This allows researchers to study exactly how one person constructs an argument, rather than how a group blends their ideas.

3. The "Gold Mine" of Annotations

Collecting the audio is just the first step. The real magic of DEBISS is what the researchers did after recording. They didn't just transcribe the words; they added layers of "metadata" (extra information) like a chef adding spices to a dish:

• The "Who Said What" Map (Speaker Diarization): They labeled exactly which student said which sentence, even when voices overlapped.
• The "Argument Map" (Argument Mining): They broke down the speech to find the "claims" (what they believe), the "evidence" (proof they used), and the "reasons" (why they believe it).
• The "Stutter Detector" (Disfluency): They marked where people said "um," "uh," or repeated themselves. This is crucial because real humans aren't perfect speakers, and AI needs to learn to understand us despite our stutters.
• The "Scorecard" (Evaluation): After the debate, students graded themselves and each other. This gives researchers a way to measure "good debating" based on human judgment, not just computer algorithms.

Why Does This Matter?

Think of DEBISS as a gym for Artificial Intelligence.

• For Language Models: It helps AI learn to understand spoken Portuguese (a language often underrepresented in tech) and how to handle the messy reality of human speech.
• For Education: It helps teachers understand what makes a student a good debater. By analyzing the data, we can see what strategies work best and help students improve their critical thinking.
• For Society: As AI becomes more common, we need to understand how humans argue about AI. This dataset captures a real conversation about AI, made by people studying AI, about the future of AI.

The Bottom Line

The researchers built a library of 9 hours and 35 minutes of real, spoken, Brazilian Portuguese debates. They cleaned it up, labeled every word, and tagged every argument. They are now opening the doors to this library so other scientists can use it to build better, more human-like AI tools.

It's not just a recording; it's a training manual for the next generation of intelligent machines, teaching them how to listen, understand, and perhaps, one day, argue back.

Technical Summary

Here is a detailed technical summary of the paper "DEBISS: a Corpus of Individual, Semi-structured and Spoken Debates."

1. Problem Statement

The field of computational debate analysis faces a significant bottleneck due to the scarcity of diverse, richly annotated corpora, particularly for spoken, semi-structured, and individual debates. Existing resources suffer from several limitations:

• Political Debate Corpora: Often too formal, adhering to strict protocols that do not reflect spontaneous interactions.
• Online Discussion Corpora: Predominantly text-based (e.g., Reddit, Twitter), lacking multimodal data (audio) and the nuances of spoken discourse (disfluencies, intonation).
• Educational Corpora: Often focus on formal academic debates or written arguments, with a notable lack of resources for Brazilian Portuguese and semi-structured formats where students defend individual viewpoints.

This gap hinders the development of specialized Natural Language Processing (NLP) models for Argument Mining (AM) and Debater Quality Analysis (DQA) in educational and non-English contexts.

2. Methodology

The authors developed the DEBISS corpus through a rigorous data collection and processing pipeline involving 67 first-semester computer science students from the Federal University of Campina Grande (UFCG), Brazil.

Data Collection

• Participants: 67 students organized into 16 groups (3–5 participants per group).
• Topic: "Generative Artificial Intelligence and its impacts on society." Participants were provided with curated news articles and a supplementary summary to ensure baseline knowledge, though reading was voluntary.
• Format: Individual, Semi-structured Debates. Unlike group-debate formats where a single perspective is defended collectively, each participant defended their own viewpoint.
• Structure: Each session followed a standardized protocol:
  1. Consent and voice sample recording.
  2. Initial Stance: Participants expressed initial thoughts.
  3. Q&A Round: The moderator asked specific questions (one per debater); others could optionally comment.
  4. Final Reflection: Mandatory closing statements.
• Environment: Recorded in a 3x5m room using a Logitech Yeti microphone (omnidirectional) and OBS Studio, capturing 9 hours and 35 minutes of audio.

Data Processing and Annotation

• Transcription: A hybrid approach was used. Three models (wav2vec2-large, Whisper-large, Azure Speech-to-Text) were benchmarked; Azure was selected as the most accurate.
• Manual Correction: Human annotators corrected automatic transcriptions using Google Sheets, ensuring accuracy and handling simultaneous speech (separating mixed audio into distinct speaker lines).
• Speaker Diarization: Each speech segment was labeled with a unique debater identifier.
• Evaluation Data:
  • Self-Evaluation: Participants completed a Google Form rating their performance, preparation, and perspective shifts (Likert scales).
  • Peer Evaluation: Participants identified the "best debater" and provided justifications.
  • Expert Evaluation: A panel of five linguistics experts evaluated the debates on criteria like Organization, Argumentation, Persuasion, and Clarity (1–5 scale), generating 80 detailed qualitative responses.

3. Key Contributions

The paper introduces the DEBISS corpus and its derivative datasets, offering several novel contributions:

• First Brazilian Portuguese Spoken Debate Corpus: Fills a critical gap in language diversity for debate analysis.
• Unique Format: Captures individual arguments within a semi-structured setting, distinct from highly formal political debates or unstructured online chats.
• Rich Multimodal Annotations: The corpus includes:
  • Speech-to-text transcriptions.
  • Speaker diarization and voice identification.
  • DEBISS-Arg: A sub-corpus annotated for Argument Mining (Argument Discourse Units, premises, claims, evidence, and micro/macro-level relations).
  • DEBISS-Eval: A sub-corpus for Debater Quality Analysis, featuring expert scores and qualitative feedback.
  • Disfluency detection labels (repetitions, hesitations).
• Educational Insight: Provides data on student oratory skill development, including self-perception vs. expert evaluation.

4. Results and Applications

The paper demonstrates the utility of DEBISS through specific use cases and statistical analysis:

• Corpus Statistics:
  • Total Duration: ~9.6 hours.
  • Total Tokens: 130,697.
  • Lexical Diversity: 0.062.
• Argument Mining (DEBISS-Arg): Successfully annotated for complex argumentative structures, enabling the study of rhetorical strategies and interaction patterns between debaters.
• Debater Quality Analysis (DEBISS-Eval): The expert evaluation data allows for the training of models to assess persuasion, clarity, and organization.
• Disfluency Detection: The dataset was used to evaluate Large Language Models (LLMs) like GPT-4o for detecting and removing speech disfluencies (repetitions, corrections). GPT-4o demonstrated strong performance, outperforming other models in maintaining text quality while cleaning the transcript.

5. Significance

• Advancement of NLP in Portuguese: DEBISS significantly expands the availability of open, annotated data for NLP research in Brazilian Portuguese, a language often underrepresented in debate corpora.
• Bridging Theory and Practice: By grounding the corpus in an educational setting, it offers insights into how argumentation skills develop among students, bridging the gap between computational analysis and pedagogical outcomes.
• Methodological Blueprint: The paper provides a reproducible methodology for collecting semi-structured spoken debates, which can be adapted for other languages and topics.
• Future Research: The authors acknowledge limitations (narrow thematic scope, homogeneous participant group) but posit that the corpus serves as a foundational resource for future work in multimodal argument mining, automated debate evaluation, and educational AI.

The dataset, including audio, transcriptions, and annotations, is publicly available via GitHub to foster further research in argumentative discourse analysis.