🤖 AI

Enhancing Human-Like Responses in Large Language Models

This paper explores techniques for enhancing the human-like qualities of large language models through improved natural language understanding, conversational coherence, and emotional intelligence, demonstrating their potential to transform user interactions and AI applications while highlighting the need to address resulting ethical concerns.

Original authors: Ethem Yağız Çalık, Talha Rüzgar Akkuş

Published 2026-02-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ethem Yağız Çalık, Talha Rüzgar Akkuş

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant, encyclopedic library assistant. This assistant knows the answer to almost every question in the world, from the capital of France to the chemical composition of a star. However, there's a catch: this assistant speaks like a robot reading a textbook. It says things like, "As an artificial intelligence, I do not have personal experiences," or "I am unable to provide that information." While accurate, it feels cold, stiff, and a bit boring to talk to.

This paper is about teaching that stiff library assistant how to chat like a warm, friendly human without forgetting anything it already knows.

Here is the story of how they did it, broken down into simple steps:

1. The Problem: The "Robot" Voice

The researchers started with powerful AI models (like Llama, Qwen, and Mistral). These models are smart, but they are trained to be formal and precise. When you ask them, "What's the best advice you've ever received?" they might reply, "I am an AI and do not receive advice." This breaks the flow of conversation. The goal was to make them sound more like a real person having a chat over coffee, while still keeping their ability to solve hard math problems or answer complex questions.

2. The Solution: Creating a "Human" Textbook

To teach the AI how to be human-like, the researchers needed a special textbook. But since they didn't have millions of real human chats to use, they built a synthetic dataset (a fake but realistic collection of data) using an even smarter AI (Llama 3 405B) as a teacher.

The Teacher's Job: The big AI was asked to generate questions (like "What's your favorite hobby?") and then write two different answers for each question.
- Answer A (The "Chosen" Human): This answer sounded natural, warm, and conversational. It might say, "Oh, I love hiking! My grandma taught me to love nature..."
- Answer B (The "Rejected" Robot): This answer sounded stiff and formal. It would say, "I am an AI language model. I do not have hobbies, but I can discuss the benefits of hiking..."
The Result: They created a massive library of 10,884 examples where the AI learned to spot the difference between a "boring robot" answer and a "friendly human" answer.

3. The Training: Teaching the AI to Choose

Once they had this special textbook, they used a technique called Direct Preference Optimization (DPO). Think of this like training a dog.

You show the dog a treat (the "Human" answer) and a rock (the "Robot" answer).
You tell the dog, "Good boy for picking the treat!"
Over time, the dog learns that the treat is what it wants.

In this case, the AI learned that the "Human" answers are the "treats" it should aim for. They used a method called LoRA (Low-Rank Adaptation), which is like giving the AI a set of sticky notes to add to its brain. Instead of rewriting its entire brain (which would be slow and risky), they just added these notes to tweak its personality. This ensured the AI didn't forget its math skills or general knowledge while learning to be friendlier.

4. The Results: Did it Work?

The researchers tested their new "Human-Like" models against the original "Robot" models.

The Human Vote: They asked real people (mostly high school students and adults) to play a game. They showed two answers side-by-side without saying which AI wrote them. The participants had to pick the one that sounded more human.
The Score: The new models won 89% to 90% of the time! People preferred the friendly, conversational answers and hated the ones that started with "As an AI..."
The Brain Test: They also tested if the AI got "dumber" by giving it hard logic and math tests. The results showed that the AI stayed almost exactly as smart as before. It just learned to talk nicer. The only slight dip was in a test specifically designed to check if the AI followed strict formatting rules (which is a small price to pay for being more natural).

5. The Catch (Limitations)

The authors are honest about the flaws in their experiment:

The "Fake" Data: Since they used AI to create the training data, it's not perfectly real. It's like learning to speak a language by reading a book written by another robot; you might miss some of the messy, chaotic nuance of real human life.
The Voters: The people who voted on the answers were mostly young students who might not speak English as their first language. This might have influenced what they considered "human-like."

6. The Warning (Ethics)

The paper ends with a serious note of caution. If an AI sounds too human, people might forget it's a machine.

Transparency: The authors argue that AI must always clearly say, "I am a robot," so people don't get tricked or form unhealthy emotional attachments.
Bias: If the AI mimics humans too well, it might accidentally copy human prejudices or biases found in the data.

Summary

In short, this paper shows that you can take a very smart but stiff AI, teach it to chat like a friendly human using a special "taste-test" training method, and make it much more enjoyable to talk to—without making it any less smart at solving problems. They have shared their new "friendly" models and the training data online for others to use.

Technical Summary: Enhancing Human-Like Responses in Large Language Models

Problem Statement
Despite significant advancements in natural language understanding driven by training on vast corpora, base Large Language Models (LLMs) such as Llama, Qwen, and Mistral Nemo often generate responses that are formal, impersonal, and lack the emotional attunement expected in human conversation. While these models excel at structured tasks, they frequently fall short in producing conversational, relatable, and emotionally intelligent interactions. This paper addresses the gap between the mechanical precision of current LLMs and the natural fluidity of human communication, aiming to enhance "human-likeness" without sacrificing accuracy in formal or structured domains.

Methodology
The authors propose a pipeline centered on the creation of specialized synthetic datasets and the application of Direct Preference Optimization (DPO) to fine-tune existing instruction-tuned models.

Synthetic Dataset Generation:
- Architecture: The team utilized Llama 3 405B for question generation and Llama 3 70B for answer generation, following a Self-Instruct approach.
- Prompt Engineering: Custom system prompts were designed to elicit two distinct response types for the same input questions:
  - Chosen: Human-like, conversational, natural, and engaging responses.
  - Rejected: Formal, impersonal, structured, and precise responses (often containing self-referential disclaimers like "I am an AI").
- Data Scope: The process generated 10,884 samples covering 256 topics (e.g., Travel, Sports, Family, Technology). The data was visualized using the Atlas Nomic Map to ensure topic diversity and balance.
- Parameters: Generation utilized a temperature of 1 and top-p of 1 to maximize creativity and variability.
Model Training:
- Techniques: The study employed Low-Rank Adaptation (LoRA) to prevent catastrophic forgetting of general knowledge, combined with DPO to implement a reward mechanism that prioritizes human-like behavior.
- Models Fine-Tuned: The methodology was applied to three open-source instruction models:
  - Llama-3-8B-Instruct
  - Qwen-2.5-7B-Instruct
  - Mistral-Nemo-Instruct-2407
- Hyperparameters: Training used a learning rate of $2 \times 10^{-4}$ , 1 epoch, and specific LoRA settings ( $r=8, \alpha=4$ ) and DPO settings ( $\beta=0.1$ ). The low $r$ value was deliberately chosen to limit weight updates, preserving the model's pre-trained general capabilities.
- Infrastructure: Training was conducted on 2x NVIDIA A100 SXM (80 GB) GPUs.

Key Contributions

Novel Dataset: The creation of a synthetic dataset specifically curated to differentiate between human-like and formal responses, serving as a foundation for DPO fine-tuning.
Fine-Tuned Models: The release of three new open-source models ("Human-Like-LLama-3-8B-Instruct," "Human-Like-Qwen-2.5-7B-Instruct," and "Human-Like-Mistral-Nemo-Instruct") available on Hugging Face.
Evaluation Framework: Implementation of an anonymous voting system via Gradio to quantitatively assess "human-likeness" against official instruction models.

Results

Human-Likeness Evaluation: In a blind voting study involving 2,000 votes across 500 questions, the fine-tuned models significantly outperformed their official counterparts.
- Human-Like-Llama-3-8B-Instruct was selected 89.6% of the time.
- Human-Like-Qwen-2.5-7B-Instruct was selected 89.5% of the time.
- Human-Like-Mistral-Nemo-Instruct was selected 79.6% of the time.
- The primary driver for this preference was the elimination of mechanical self-disclaimers (e.g., "As an AI...") and improved conversational flow.
Benchmark Performance: Evaluation on the Open LLM Leaderboard (IFEval, BBH, MATH, GPQA, MuSR, MMLU-Pro) showed minimal performance degradation.
- Most models exhibited slight reductions in average scores, primarily driven by the IFEval benchmark.
- When IFEval was excluded, some models (Qwen and Mistral variants) showed slight improvements or negligible changes, indicating that the "human-like" tuning did not significantly compromise general reasoning or task-specific accuracy.

Significance and Claims
The paper claims that open-source models can be effectively fine-tuned to produce responses that are more conversational and closely resemble human communication, directly addressing the issue of formal and impersonal outputs. The authors emphasize that this enhancement in naturalness is achieved without a noticeable loss in accuracy or efficiency across standard benchmarks.

The study highlights the successful integration of psychological insights and synthetic data generation to bridge the gap between AI and human interaction. By releasing the models and the dataset, the authors aim to provide resources for future research into making AI systems more engaging and effective in real-world applications. The work also acknowledges the ethical necessity of transparency, noting that as models become more human-like, clear disclosure of their machine nature is critical to prevent user confusion and maintain trust, aligning with frameworks like the EU AI Act.