Technical Summary: From Classification to Ranking for MBTI Personality Detection

Problem Statement

Personality detection, particularly within the Myers-Briggs Type Indicator (MBTI) framework, traditionally treats the task as four independent binary classification problems (E/I, S/N, T/F, J/P). The authors argue this approach suffers from two critical shortcomings:

1. Dimensional Decoupling: It ignores the psychometrically established hierarchical interactions and dynamic interdependencies between dimensions (e.g., how a dominant cognitive function like Introverted Intuition shapes the expression of other traits).
2. Static Labeling: It fails to capture the continuous spectrum of personality, neglecting the relative position of an individual on a trait scale.

Furthermore, existing Large Language Model (LLM) approaches, such as prompt-based Chain-of-Thought (CoT) methods (e.g., PsyCoT), rely heavily on expert-crafted prompts and large-scale models. This creates high inference latency, computational costs, and a lack of autonomous pattern-learning capacity, making them impractical for real-time applications.

Methodology: PerDet-R1

The paper proposes PerDet-R1, a two-stage framework that reframes personality detection from a categorical classification task to a list-wise ranking task. The goal is to rank the most relevant MBTI types for a user based on their social media posts, rather than assigning a single discrete label.

1. Training Data Construction

The dataset construction involves a rigorous two-stage process to adapt data for different fine-tuning objectives:

• Stage 1 (SFT Data): The authors use the Qwen-plus model as a teacher to generate reasoning traces and Top-K answer lists from social media posts. They employ rejection sampling to filter out samples where the ground-truth label does not appear in the model's Top-K predictions. This ensures high-quality, readable inference chains.
• Stage 2 (RL Data): The original dataset (excluding the SFT samples) is used for Reinforcement Learning to prevent data leakage and ensure the model learns from the full distribution of behavioral data.

2. Stage 1: Supervised Fine-Tuning (SFT)

The base model (Qwen2.5-7B-Instruct) undergoes supervised fine-tuning to distill reasoning capabilities from the teacher model (Qwen-plus).

• Objective: To instill foundational reasoning patterns and domain knowledge without over-emphasizing immediate performance.
• Output: The model learns to output an ordered list of Top-K MBTI types with explicit awareness of dimensional combinations and interactions, formatted as <answer>[option 1, option 2, ...]<answer>.

3. Stage 2: Reasoning Reinforcement Learning (GRPO)

The model is further optimized using Group Relative Policy Optimization (GRPO), a value-free algorithm that computes normalized group-level advantages.

• Reward Function Design: To address the sparsity and lack of differentiation in standard classification rewards, the authors design a specialized ranking-based reward function combining:
  1. Normalized Discounted Cumulative Gain (NDCG): Measures the relevance and position of the correct answer in the output list, prioritizing higher ranks.
  2. Dimension Similarity (DS) Reward: Calculates similarity based on the four MBTI dimensions. The similarity score $s(\hat{t}, t^*)$ is computed by summing weighted character matches across the four dimensions, with a hyper-parameter $\epsilon$ controlling the weight of each dimension's contribution.
• Total Reward: The sum of NDCG and DS rewards. This design aims to amplify subtle trait interactions and avoid "reward hacking" by ensuring the model learns optimal answer rankings rather than just correct classifications.

Key Contributions

1. Paradigm Shift: The paper introduces the first list-wise ranking paradigm for personality detection, eliminating the "categorical fallacy" of treating MBTI dimensions as independent binary tasks.
2. Novel Reward Modeling: A trait-cohort advantage estimation is introduced for the reward model. By calculating rewards based on dimension similarity and NDCG, the method generates dense reward signals that capture cross-dimensional interactions and mitigate reward hacking.
3. State-of-the-Art Performance: The proposed method achieves superior results on mainstream benchmarks, establishing a new standard for data-driven personality assessment that reduces dependency on expert-crafted prompts.

Experimental Results

Experiments were conducted on the Kaggle and PANDORA datasets. The model was evaluated using Macro-F1 (for binary 4-dimension classification), F1-Score (for 16-type multiclass classification), and NDCG@k (for ranking).

• Performance Gains: PerDet-R1 outperformed all baselines, including traditional ML models (SVM, XGBoost), BERT-based architectures (D-DGCN), and LLM-based methods (PsyCoT, ETM).
  • On the Kaggle dataset, PerDet-R1 achieved a Macro-F1 of 80.57% (vs. 77.79% for the previous best, ETM) and an F1-score of 41.34% (vs. 32.55%).
  • On the PANDORA dataset, it achieved a Macro-F1 of 66.10% and an F1-score of 35.08%.
• Ablation Studies:
  • Removing the two-stage training (SFT + GRPO) caused significant performance drops (e.g., NDCG dropped by ~20-24%).
  • Replacing the ranking strategy with classification resulted in a 17.50% decrease in binary F1, confirming the superiority of the ranking approach.
  • Both NDCG and Dimension Similarity rewards were found to be essential; removing either caused reward signals to collapse or performance to degrade.

Significance and Claims

The authors claim that their work fundamentally reconceptualizes personality detection as a comparative ranking problem rather than disjoint binary classifications. This shift allows the model to:

• Capture the nuanced trait interactions and hierarchical cognitive functions inherent in MBTI theory, which are obscured by traditional classification methods.
• Achieve autonomous pattern learning from behavioral data, reducing reliance on expensive expert prompts and large-scale parameter models.
• Provide a robust initialization and generalization capability through the combination of off-policy distillation (SFT) and position-sensitive reinforcement (GRPO).

The paper concludes that this framework offers a more holistic and reasoning-driven approach to personality modeling, achieving state-of-the-art results while addressing the inherent complexity and blurred boundaries of human personality traits. Future work is suggested to explore multi-modal data (e.g., avatars, facial expressions) to further enhance detection performance.