🤖 AI

Toward Anthropomorphic Dialogue: A Closed-Loop Framework for Human-Like Chat Generation, Evaluation, and Preference Alignment

The paper introduces AnthroDial, a closed-loop framework that unifies role-conditioned dialogue generation, executable multi-dimensional evaluation, and cognitive-diagnostic preference alignment to significantly improve human-like chat performance across diverse personas and scenarios.

Original authors: Wentao Liu, Siyu Song, Xi Chen, Youjia Li, Xiaokun Wang, Min Ji, Ji Wang

Published 2026-07-21✓ Author reviewed ⓘ

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Wentao Liu, Siyu Song, Xi Chen, Youjia Li, Xiaokun Wang, Min Ji, Ji Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to be a friend. For a long time, scientists have been great at teaching robots to speak fluently, follow instructions, and answer questions politely. But there is a big difference between a robot that answers correctly and a robot that feels like a real person you could text late at night. A real friend doesn't just give you a perfect dictionary definition of "sadness"; they remember what you said yesterday, they know when to send a quick "ouch" and when to wait a few minutes before replying, and they know exactly how much personal information to share based on how close you are. This paper lives in the world of artificial intelligence, specifically the branch trying to make chatbots feel less like helpful assistants and more like human companions. The core idea is that being "human-like" isn't just about having a cool personality; it's about managing a complex web of rules: remembering your history, respecting the timing of a text message, knowing your own limits, and keeping the conversation moving naturally without getting stuck or repeating yourself.

The researchers behind this paper, who call their system AnthroDial, realized that current chatbots often fail at this "private chat" vibe. They might sound too formal, forget facts they just learned, or act like a customer service agent even when you're just venting about a bad day. To fix this, they didn't just tweak the robot's personality; they built a complete "closed-loop" factory. Think of it like a video game where the robot plays against itself, gets graded by a strict coach, learns from its mistakes, and tries again. This factory has three main parts: a scheduler that decides when to type and what to draft (like a human pausing to think before hitting send), a judge that checks if the robot is breaking the rules of being human (like claiming to see you in person when you're miles apart), and a training coach that uses a special scoring system to tell the robot exactly which skills it needs to practice the most.

The paper's main finding is that this "closed-loop" approach actually works. They tested their system against 16 different AI models, including some of the biggest and smartest ones available. The results suggest that simply making a model bigger or letting it "think" longer before answering isn't enough. Instead, training the model specifically on these human-like behaviors—using their special factory—made a huge difference. Their best trained model, a 27-billion-parameter system, reached a strict "passing grade" of 39.00% on their difficult test, beating the best untrained models which only scored 32.00%. Even more surprisingly, they showed that this training could be squeezed into a much smaller, faster model (9 billion parameters) that doesn't use extra "thinking" time. This smaller model jumped from a 0.00% pass rate to 18.37% after training, proving that the "human-like" behavior is something the model can actually learn and keep, rather than just faking it with extra computing power.

The paper explicitly argues against the idea that "fluent" speech equals "human-like" conversation. They show that a robot can be grammatically perfect and still fail miserably at being a friend if it forgets the relationship, ignores the time of day, or acts like a therapist when you just want a buddy. They also rule out the idea that you can just add a "persona" prompt to a standard chatbot and expect it to work; without the specific architecture to manage memory, timing, and boundaries, the robot inevitably slips back into being a generic assistant.

In their experiments, the team built a massive testing ground with 55 different character profiles (like a shy student or a busy parent) and 50 different scenarios (like a work setback or a pet emergency). They ran 100 test cases for each model. The results were measured with a very strict "all-or-nothing" rule: if the robot made even one small mistake—like repeating a sentence or forgetting a fact—the whole conversation was marked as a failure. Under these strict rules, the untrained models struggled mightily. For instance, a popular 9-billion-parameter model without any special training got a 0.00% pass rate. But after going through the AnthroDial training pipeline, that same small model improved to 13.00% with just standard training, and 18.37% with their advanced "reward" training. The big 27-billion model, when fully trained, reached 39.00%.

The authors suggest that their secret sauce is a "diagnostic reward" system. Imagine a teacher who doesn't just give you a final grade of "B," but instead gives you a report card that says, "You are great at math, but you need to practice your spelling." Their system does this for every single skill a chatbot needs: remembering facts, keeping a natural rhythm, staying in character, and knowing when to stop talking. It uses a mathematical trick (called a Kalman filter) to track how good the robot is at each skill and then focuses the training on the skills the robot is worst at. This ensures the robot doesn't just get better at what it's already good at, but actually fixes its weak spots.

One of the most playful parts of their system is how it handles time. Real people don't reply instantly every time; sometimes they take a break, sometimes they send a quick "haha," and sometimes they draft a message, wait, and then change their mind. AnthroDial treats this like a "single-draft scheduler." The robot doesn't just spit out a message; it drafts it, sets a timer, and decides if it should wait, interrupt, or close the conversation. This makes the chat feel like it has a heartbeat, with pauses and rhythms that match a real human texting on WeChat.

The paper also highlights that being "proactive" (starting a new topic or asking a question) is tricky. A robot that asks "What happened next?" every five minutes isn't being proactive; it's being annoying. The training taught the models to be proactive in smarter ways, like sharing a small personal thought, making a gentle joke, or suggesting a break. The results showed that while the models got much better at these subtle social cues, they still struggle with the hardest topics. For example, in "technology" and "finance" scenarios, even the best trained model only passed the strict test 0.0% and 12.5% of the time, respectively. This suggests that while the system is a huge step forward, there are still very difficult areas where the robot hasn't quite learned to act human yet.

Ultimately, the paper concludes that making a chatbot feel human is a system problem, not just a language problem. You can't just write a better prompt; you need to build a system that manages memory, time, and social boundaries, and then train it with a reward system that specifically targets the robot's weaknesses. The authors are careful to note that this is about behavior, not about the robot actually having feelings or consciousness. They emphasize that the goal is to create a text behavior that feels real, not to trick people into thinking the robot is a human. In fact, they argue that a truly "human-like" system should be honest about being an AI, just like a very good actor who stays in character but never claims to be the character they are playing. The work suggests that with the right training loop, we can get much closer to that goal, turning cold, robotic responses into warm, natural conversations that feel like they are coming from a real friend.

Technical Summary: Toward Anthropomorphic Dialogue (AnthroDial)

1. Problem Formulation

The paper identifies a critical gap in current open-domain dialogue systems: while models have become fluent and cooperative, they often fail to simulate a plausible human interlocutor in private, multi-turn conversations. Standard models tend to over-explain, maintain an assistant-like posture, forget established facts, shift personas, or violate medium-specific constraints (e.g., timing, silence).

The authors define anthropomorphic dialogue not as a claim of consciousness or emotion, but as a behavioral constraint satisfaction problem. A system must generate text that adheres to observable constraints under specific conditions:

Persona: Identity, age, occupation, personality, and knowledge boundaries.
Relationship: The specific dynamic between speakers (e.g., friends vs. colleagues).
Memory: Long-term retention of preferences and facts, distinct from immediate context.
Medium: Constraints of private messaging (e.g., WeChat style, timing, rhythm).
Time: Virtual time state and pacing.

The core challenge is that these dimensions are often treated as flat style prompts rather than runtime states, leading to failures in coherence, initiative, and naturalness over long interactions.

2. Methodology: The AnthroDial Framework

The authors propose AnthroDial, a closed-loop framework comprising three coupled components: a role-conditioned runtime, an executable benchmark, and a diagnostic alignment pipeline.

2.1 Role-Conditioned Scheduled Dialogue Runtime

Unlike standard turn-by-turn generation, AnthroDial models private chat as stateful scheduling.

State Representation: The system maintains persona cards, scenario cards, partner-specific long-term memory, compressed conversation history, and a virtual clock.
Single-Draft Decision: At each step, the model outputs a JSON decision tuple $y_t = (u_t, \Delta t_t, e_t)$ , where $u_t$ is the message content, $\Delta t_t$ is the planned send delay, and $e_t$ is a silence or closure decision.
Scheduling Logic: A scheduler manages a "Draft Buffer." It emits the earliest pending draft, allowing for multi-message bursts, interruptions, and natural pauses. This exposes behaviors like waiting after a question or dropping a stale draft, which are hidden in standard generation.
Memory Management: Long-term memory is updated via a separate module ( $U_\phi$ ) to retain durable facts while discarding transient events. Short-term memory uses a sliding window with a concise summary for older turns.

2.2 Executable Benchmark

The framework introduces a hierarchical, executable benchmark to evaluate behavior rather than just aggregate quality.

L0 Validity Gate: A deterministic filter that rejects samples with fatal failures (e.g., AI identity leakage, impossible medium behavior, hard repetition, role contradiction).
Behavioral Dimensions: Valid samples are scored on ten dimensions:
- Local (Turn-level, D1–D5): WeChat naturalness, conciseness, persona consistency, knowledge boundaries, and rhythm/segmentation.
- Global (Dialogue-level, E1–E5): Context consistency, scenario/relationship fit, empathic/social fit, proactivity, and dialogue-arc usability.
Scoring: The system reports a weighted score and a Strict ACC (acceptance rate), where a case passes only if all checkboxes in all turns and the holistic card are satisfied.

2.3 Closed-Loop Training Pipeline

The framework generates its own training data and uses diagnostic signals for reinforcement learning.

Data Synthesis: The runtime generates dialogues which are filtered. Accepted dialogues form the SFT pool; rejected dialogues are converted into failure records with repair suggestions.
SFT: Supervised Fine-Tuning is performed on 16,436 scheduled-decision examples, training the model on both linguistic style and private-message timing (delays, closures).
CDT-ZPD Guided GRPO: The Reinforcement Learning stage uses Group Relative Policy Optimization (GRPO) with a novel reward mechanism designed to address uneven capability growth:
- Kalman Filtering: Maintains a latent capability estimate ( $\theta_{k,t}$ ) for each of the 10 behavioral dimensions.
- Capability-Deficit Weighting: Increases reward weight for dimensions where the model's estimated capability is low ( $d_{k,t} = \epsilon + (1 - \theta_{k,t})^p$ ).
- ZPD (Zone of Proximal Development) Matching: Uses the current rollout's dimension score as an estimate of task attainability. The reward is modulated to focus optimization on tasks that are "learnable" (close to the current capability estimate) rather than too easy or impossibly hard.
- Adaptive Reward: The final reward combines a legacy rubric score with this adaptive, diagnostic signal, shifting optimization pressure toward weak but learnable human-like skills.

3. Key Contributions

Formalization of Anthropomorphic Dialogue: Defined as conditional multi-turn generation under persona, scenario, memory, medium, and time constraints, instantiated via a scheduled private-chat architecture.
Executable Benchmark: Designed with an L0 validity gate and separate local/global behavioral dimensions, allowing for the reporting of invalid samples, graded quality, and specific failure dimensions.
Closed-Loop Training Pipeline: A system that uses 16,436 SFT samples and a CDT-ZPD guided GRPO objective. It utilizes Kalman-filtered capability estimates and ZPD matching to turn evaluation traces into adaptive rewards.
Empirical Benchmarking: A comprehensive snapshot of 16 systems (including frontier baselines, open models, and ablations) evaluated on 55 personas, 50 scenarios, and 100 role-conditioned cases per model.

4. Results

The paper evaluates models across different scales (9B, 27B, 35B, 397B) and inference modes (Thinking vs. No-Think).

Baseline Performance: The strongest non-trained baseline (Gemini-3.5-Flash) achieves 32.00% strict ACC.
SFT + RL Performance (27B): The full Qwen3.6-27B-SFT+RL system (evaluated in thinking mode) achieves 39.00% strict ACC and an overall score of 98.5. This represents a +7.00 absolute point improvement over the strongest non-trained baseline and a +20.00 point improvement over the 27B SFT-only variant.
Efficiency Track (9B No-Think): In the low-latency setting without test-time thinking:
- Untuned 9B model: 0.00% strict ACC.
- 9B-SFT: 13.00% strict ACC.
- 9B-SFT+RL: 18.37% strict ACC.
- Significance: The 9B SFT+RL model outperforms the untrained 27B thinking model in strict ACC, demonstrating that post-training can transfer anthropomorphic behavior to smaller, faster models.
Dimensional Analysis: Post-training significantly improves global dimensions (context consistency, scenario fit) which are often weak in standard models. The 9B model excels in local dimensions (naturalness, rhythm) but lags in global coherence, highlighting the divergence between local fluency and global anthropomorphic quality.

5. Significance and Claims

The paper claims that anthropomorphic dialogue is a system property requiring a closed loop of generation, evaluation, and reward shaping that shares the same behavioral dimensions.

Beyond Fluency: High per-turn naturalness does not guarantee a convincing conversation; global coherence and adherence to constraints are distinct challenges.
Training vs. Inference: Post-training (SFT and RL) contributes learned anthropomorphic behavior that is not merely a result of test-time "thinking" or deliberation. The pipeline successfully distills these behaviors into smaller, low-overhead models.
Diagnostic Alignment: The CDT-ZPD reward mechanism effectively targets weak skills by dynamically adjusting reward weights based on the model's current capability estimates, rather than optimizing for a single aggregate score.
Medium Specificity: The current instantiation focuses on Chinese private-messaging (WeChat-style), where medium-specific norms (timing, rhythm, brevity) make failures observable. The framework is designed to be extensible to other mediums by adjusting the surface norms while preserving the behavioral evaluation structure.

The authors maintain a modest stance, noting that the results evaluate the pipeline and benchmark snapshot rather than a deployed product, and that automatic judges may have biases. The primary contribution is a structured methodology for measuring and aligning human-like conversational behavior without claiming the system possesses genuine consciousness or emotion.