Technical Summary: StagePilot

Problem Statement

Cybergrooming represents a critical and evolving threat to youth, where predators exploit digital platforms to manipulate minors, build trust, and coerce them into sexual exploitation. While existing research has focused on detecting malicious chats after damage occurs or analyzing victim traits, there is a significant gap in proactive, victim-centric prevention tools. Current simulation-based training lacks realistic, strategic predator-victim interactions that reflect the nonlinear, stage-wise progression of grooming. Furthermore, directly involving human participants (minors) in such simulations raises severe ethical concerns, necessitating a safe, controlled, and reproducible alternative for educational training.

Methodology

The authors propose StagePilot, an offline Deep Reinforcement Learning (DRL) agent designed to simulate stage-controlled cybergrooming conversations. Unlike traditional dialogue systems that generate text turn-by-turn, StagePilot reframes the problem as a planning task over abstract interaction stages.

Core Framework

1. Stage-Controlled Formulation: The system adopts O'Connell's six-stage typology of cybergrooming (Friendship Forming, Relationship Forming, Risk Assessment, Exclusivity, Sexual Solicitation, and Conclusion). The agent learns a policy to select the next stage rather than generating raw utterances.
2. Hybrid Architecture:
   • RL Agent: Operates as a stage-level planner. It observes the conversational context (last four utterances and the previous stage) and selects the next stage action.
   • LLM Environment: Large Language Models (specifically Mistral 7B Instruct v0.2) act as the environment dynamics. They generate the actual textual responses for both the "predator" and the "victim" conditioned on the stage selected by the RL agent.
3. Offline Learning: Due to the ethical impossibility of online RL with minors, the agent is trained offline on the Perverted-Justice (PJ) dataset. The dataset consists of chat logs between predators and decoys, annotated with the six grooming stages using an ensemble of LLMs (GPT-4o and Claude 3.5 Sonnet).
4. Constraints and Rewards:
   • Transition Constraints: To ensure behavioral plausibility, the action space is masked to allow only transitions to adjacent stages (e.g., Stage 2 $\to$ 1, 2, or 3), preventing unrealistic jumps.
   • Composite Reward: The reward function balances two objectives:
     • Sentiment Reward ($r_{sent}$): Derived from a RoBERTa-based classifier, measuring the positivity of the victim's response to maintain engagement.
     • Distance Reward ($r_{dist}$): A linear function incentivizing progression toward the final stage (Stage 6).
     • Total Reward: $R_t = \alpha \cdot r_{sent} + \beta \cdot r_{dist}$.

Algorithms Evaluated

The study benchmarks several policy learning methods under uniform constraints:

• Prompt Engineering: Zero-shot prompting without fine-tuning.
• Behavior Cloning (BC): Supervised learning mimicking observed stage transitions.
• Conservative Q-Learning (CQL): Offline RL penalizing out-of-distribution actions.
• Implicit Q-Learning + Advantage-Weighted Actor-Critic (IQL+AWAC): A hybrid approach combining conservative value estimation with reward-sensitive policy updates.

Key Contributions

1. Stage-Controlled Dialogue Formulation: The paper introduces an offline RL framework where dialogue flow is controlled via abstract interaction stages rather than direct text generation, enabling interpretable, long-horizon planning.
2. Behaviorally Grounded Transition Constraints: The implementation of adjacent-stage masking ensures that stage progressions are flexible yet behaviorally plausible, mirroring real-world grooming dynamics while preventing abrupt, unrealistic jumps.
3. Emotion-Aware Composite Reward: A novel reward design integrates victim sentiment signals with stage-distance goals, enabling the agent to balance emotional engagement with strategic progression.
4. Comprehensive Evaluation Framework: The authors establish a controlled evaluation covering stage reachability, transition dynamics, sentiment alignment, and dialogue efficiency, comparing zero-shot, supervised, and offline RL baselines.
5. Empirical Validation: The study demonstrates that the proposed framework effectively unifies emotional coherence with strategic planning, achieving superior performance in reaching final grooming stages compared to baselines.

Experimental Results

The system was evaluated through 100 simulated dialogues per model, comparing performance across stage progression, sentiment alignment, and efficiency.

• Performance Superiority: The IQL+AWAC agent achieved the best balance between strategic planning and emotional coherence. It reached Stage 6 in 95% of simulations and completed successful terminations (5+ turns in Stage 6) in 91% of cases.
• Comparison with Baselines:
  • Prompt Engineering performed poorly, reaching Stage 6 in only 14% of cases, often looping in early stages.
  • Behavior Cloning (BC) reached Stage 6 in 52% of cases but struggled with long-horizon planning, resulting in inefficient trajectories (average 130.8 turns for successful runs).
  • CQL improved upon BC (83% Stage 6 reach) but showed a trade-off with sentiment, generating more negative victim responses.
• Efficiency: IQL+AWAC generated the shortest successful conversations (average 64.89 turns) while maintaining over 70% sentiment alignment (positive or neutral), demonstrating efficient, goal-directed planning.
• Transition Dynamics: RL-based methods exhibited higher forward transition rates and lower stagnation compared to baselines. IQL+AWAC specifically showed a strategic ability to navigate later stages (5 and 6) while minimizing time in early rapport-building stages when necessary for goal completion.
• Ablation Studies: The research highlighted a critical trade-off between sentiment and progression. Over-weighting sentiment rewards ($\alpha \ge 0.8$) led to repetitive early-stage loops, while distance-based incentives ($\beta$) drove efficient progression. The selected configuration ($\alpha=0.8, \beta=0.2$ with DistilRoBERTa) offered the best pedagogical balance.

Significance and Claims

The paper claims that StagePilot provides a safer, controlled alternative for real-time preventive education by simulating grooming dynamics without exposing minors to harm. Its significance lies in:

• Ethical Safeguarding: By using fully automated, simulated interactions (predator and victim LLMs), the framework adheres to Responsible AI principles, allowing for the exploration of conversational dynamics before any human-in-the-loop studies.
• Pedagogical Utility: The system generates realistic and coherent conversations aligned with grooming dynamics, offering a tool to strengthen teenagers' recognition of grooming red flags and response strategies.
• Methodological Advancement: It demonstrates that offline DRL with stage-level abstraction is superior to imitation learning and rule-based approaches for safety-critical, long-horizon dialogue simulation.

The authors explicitly state that StagePilot is not intended for real-world deployment involving children and should only be used under strict ethical oversight. They emphasize that the simulation is more temporally compressed and linear than typical real-world cases, serving as a formative educational tool rather than a perfect replica of every real-world interaction. The work concludes that stage-level planning is a viable mechanism for structuring safety-critical interactions, with potential extensions to counseling simulations and negotiation role-plays.