PersonaKit (PK): A Plug-and-Play Platform for User Testing Diverse Roles in Full-Duplex Dialogue

Technical Summary: PersonaKit (PK)

Problem Statement
As spoken dialogue systems transition from half-duplex to full-duplex interactions, the design frontier has shifted from text quality to sociolinguistic behavior, specifically the management of overlapping speech (barge-in). Current full-duplex systems predominantly default to a rigid "always-yield" policy, where the agent immediately cedes the floor upon detecting user speech. While appropriate for subservient assistants, this behavior breaks psychological immersion for non-submissive personas (e.g., authoritative instructors, uncooperative merchants) who would naturally interrupt or hold the floor in human conversation. Evaluating alternative, persona-specific turn-taking strategies is currently hindered by the substantial engineering overhead required to build real-time full-duplex test environments involving WebRTC, Voice Activity Detection (VAD), dynamic prompt injection, and latency tracking.

Methodology and System Architecture
To address these bottlenecks, the authors present PersonaKit (PK), an open-source, low-latency web platform designed for the rapid prototyping and evaluation of conversational agents. The system isolates low-latency audio engineering from experimental design, allowing researchers to configure experiments entirely through a web dashboard and four JSON files without modifying source code:

• persona.json: Defines the scenario, role, and opening prompt.
• interruption_config.json: Specifies a strategy matrix mapping user interruption intents to agent actions.
• session_config.json: Configures post-session surveys.
• model_config.json: Routes generation and Text-to-Speech (TTS) to specific providers.

Core Technical Components:

1. Client-Side VAD and Audio Tracking: The frontend utilizes WebRTC with a client-side VAD node to halt local playback immediately upon detecting a "barge-in." It tracks byte-level playback to log the exact "cutoff text" (what the bot vocalized before interruption) and the "remaining text" (what it intended to say), transmitting this precise context to the server.
2. Intent Classification and Turn-Taking Manager: Upon interruption, the server transcribes the user's utterance and classifies its intent into four categories grounded in conversation analysis: Competitive (seeking the floor), Cooperative (adding info), Topic Change, and Backchannel.
3. Probabilistic Strategy Injection: Researchers define a strategy matrix in interruption_config.json mapping these intents to four actions: Yield, Resume/Hold, Bridge, or Override, with associated probability weights. The Turn-Taking Manager samples an action from this distribution and injects it as a control token (e.g., [STRATEGY=RESUME]) into the LLM's system prompt. This forces the LLM to generate a response conditioned on a pre-committed action rather than choosing a strategy zero-shot.
4. Automated Lifecycle: Sessions terminate based on turn limits or verbal exit intents. The system automatically triggers a post-session survey and exports structured logs (transcripts, event logs with intent/strategy data, and survey responses) in JSON or CSV formats.

Pilot User Study and Results
The authors conducted a pilot study with five participants engaging with eight distinct personas mapped to the Interpersonal Circumplex (varying in Agency and Communion). Each participant experienced three conditions per persona:

• Style A (Baseline): Always-Yield.
• Style B (Probabilistic): JSON-tuned strategy weights.
• Style C (Autonomous): LLM selects strategy zero-shot.

Key Findings:

• Persona-Dependent Preferences: High-agency personas (e.g., Drill Sergeant) showed improved Reaction Naturalness with non-yielding strategies (0.60 for Probabilistic vs. 0.20 for Always-Yield), with 60% of participants preferring the Autonomous condition. Conversely, low-agency, high-communion personas (e.g., AI Assistant) favored the Always-Yield policy (70% preference).
• Emergent Behaviors: The probabilistic approach enabled coherent barge-in recoveries that the Always-Yield baseline would discard. For example, a Drill Sergeant interrupted mid-sentence successfully resumed the sentence ("...again!") when the system sampled a "Resume" action, maintaining character consistency.
• Qualitative Feedback: Participants noted that non-yielding strategies felt more "human-like" for authoritative roles, while yielding strategies were preferred for service roles.

Key Contributions

1. Open-Source Platform: A plug-and-play web platform for full-duplex persona evaluation that eliminates the engineering bottleneck of building custom real-time dialogue environments.
2. Configurable Turn-Taking: A JSON-based mechanism that elevates persona-conditioned interruption policies to first-class configurable objects, allowing researchers to define probabilistic turn-taking behaviors without code changes.
3. End-to-End Workflow: An integrated system supporting live deployment, automated comparative A/B surveys, and structured data export for analyzing complex sociolinguistic behaviors.

Significance and Limitations
The paper claims that PersonaKit provides an extensible, end-to-end framework for studying how turn-taking strategies affect user perception and persona consistency in next-generation spoken agents. It demonstrates that preferred interruption policies vary significantly based on the persona's role within the Interpersonal Circumplex.

The authors maintain a modest stance regarding their findings. They acknowledge that the pilot study (N=5) is descriptive rather than inferential, requiring larger samples for robust claims about circumplex-to-strategy mappings. They also note limitations in their zero-shot intent classification (which may mislabel ambiguous back-channels) and the exclusion of fine-grained prosodic cues (pitch, gaze) in favor of configurability. The system is presented as a testbed for future research rather than a finalized commercial solution.