Personality as Relational Infrastructure: User Perceptions of Personality-Trait-Infused LLM Messaging

1. Problem Statement

Digital behavior change systems (e.g., for physical activity) increasingly rely on Large Language Models (LLMs) to generate personalized, Just-In-Time Adaptive Interventions (JITAIs). While current research focuses on message-level optimization (improving the immediate receptivity of a single message), it remains unclear whether personalization works through:

1. Trial-level effects: Improving the evaluation of individual messages in isolation.
2. Person-level effects: Shaping overall user perceptions through cumulative exposure patterns over time.

Furthermore, there is a lack of empirical evidence comparing different LLM generation strategies (e.g., fine-tuning vs. prompting) in the context of personality-based personalization. The authors hypothesize that personality-based adaptation may function as a "relational infrastructure," where consistent exposure builds rapport (similar to human relationships) rather than optimizing discrete interactions.

2. Methodology

Study Design

The research employed a controlled, retrospective, within-subjects design involving 90 participants.

• Scenario: Participants recalled five distinct physical activity scenarios from the past week, providing contextual data (mood, location, activity, barriers).
• Task: Participants evaluated LLM-generated messages they would have received in those situations.
• LLM Configurations: The study compared four generation strategies, each implemented with and without Big Five Personality Traits (BFPT) integration (8 total configurations):
  1. Baseline: Minimal prompting.
  2. Few-Shot Chain-of-Thought (FS-CoT-SC): Using self-consistency and user feedback examples.
  3. Retrieval-Augmented Generation (RAG): Retrieving similar context examples from a training set.
  4. Fine-Tuning: Models trained on the specific dataset.
• Personality Integration: BFPT data (Openness, Conscientiousness, Extraversion, Agreeableness, Neuroticism) was used to adapt tone and framing. The LLM independently decided whether to send a message based on context, mimicking real-world JITAI logic.

Data Collection & Measures

• Participants: 90 adults (balanced gender, mean age ~37) recruited via Prolific.
• Metrics:
  • Primary: Perceived Personalization (Relevance, Fit, Generic content).
  • Secondary: JITAI Quality (Appropriateness, Engagement, Effectiveness, Professionalism) and Affective Response (Positive/Negative emotions).
• Statistical Approach: The authors used Bayesian Multilevel Ordinal Regression with within-between decomposition. This allowed them to separate:
  • Trial-level effects: Does a specific BFPT message rate higher than a non-BFPT message for the same user?
  • Person-level effects: Do users who received a higher proportion of BFPT messages rate the system more favorably overall?
• Controls: Confounding variables (health status, activity levels, participant type) were controlled via multicollinearity analysis (VIF) and inclusion in the model.

3. Key Results

A. Dissociation Between Trial and Person Levels

• Trial-Level (Null): There was no credible association between receiving a personality-informed message and the immediate rating of that specific message. The 97% Highest Density Intervals (HDIs) for trial-level effects crossed zero for all outcomes.
• Person-Level (Significant): Participants who received a higher proportion (dosage) of BFPT-informed messages across the study rated the system significantly higher on:
  • Personalization: Higher relevance and fit; lower perception of generic content.
  • Quality: Higher scores for appropriateness, professionalism, effectiveness, and engagement.
  • Affect: Reduced negative emotions (anger, sadness, fear, frustration).
  • Note: Positive emotions (happiness) showed a positive trend but were not statistically significant in all models.

B. Variance Partitioning (Intraclass Correlation)

• Between-Person Dominance: Intraclass Correlation Coefficients (ICCs) ranged from 54% to 81%, indicating that the majority of variance in user ratings was due to stable individual differences (baselines) rather than fluctuations between specific messages.
• Exception: "Empathy" showed a lower ICC (23%), suggesting it is more sensitive to momentary message features than stable exposure.

C. LLM Generation Strategies

• No Strategy Advantage: There were no credible differences in performance between the four LLM strategies (Baseline, Few-Shot, Fine-Tuned, RAG).
• Resource Efficiency: Simpler prompting methods performed equivalently to resource-intensive methods (Fine-tuning, RAG), suggesting that the consistency of personality integration matters more than the sophistication of the generation architecture.

D. Dose-Response Relationship

• Analysis of exposure quartiles (40% vs. 57% BFPT exposure) revealed a dose-response effect: higher exposure to personality-aligned messages correlated with progressively better ratings for positive outcomes and lower negative affect.

4. Key Contributions

1. Empirical Evidence of Relational Infrastructure: The study provides evidence that personality-based personalization in LLMs operates primarily through aggregate exposure (cumulative relationship building) rather than per-message optimization. This challenges the prevailing "one-shot" optimization paradigm in digital health.
2. Relational Infrastructure Concept: The authors introduce "Relational Infrastructure" as a design concept, defining stable user features (like BFPT) that scaffold consistent system accommodation across interactions, shifting focus from isolated message quality to the durability of the accommodation stance.
3. LLM Strategy Equivalence: The finding that simple prompting matches fine-tuned models for personality-infused interventions offers practical guidance for deploying scalable, resource-efficient AI in healthcare.
4. Methodological Innovation: The use of Bayesian multilevel modeling with within-between decomposition successfully isolates immediate message effects from cumulative exposure effects, a distinction often overlooked in HCI and health intervention studies.

5. Significance and Implications

• Design Implications: Adaptive systems should prioritize consistency and long-term exposure to personality-aligned communication over optimizing individual message content. Designers should view personality traits as a stable "infrastructure" for building trust rather than a variable for immediate click-through optimization.
• Evaluation Frameworks: Current evaluation metrics focusing on immediate response (e.g., A/B testing single messages) may fail to detect the true value of personalization. Future studies must incorporate longitudinal designs to capture cumulative relationship effects.
• Theoretical Framework: The results align with Communication Accommodation Theory (CAT), suggesting that linguistic convergence builds rapport cumulatively. The study posits that AI systems can foster "Human-AI Relations" through consistent, low-saliency adaptation that users may not consciously detect in single interactions but appreciate over time.
• Transparency: The findings raise questions about transparency; if users benefit from personality alignment without consciously noticing it in individual messages, ethical considerations regarding informed consent and "AI mimicry" need further exploration.

Limitations & Future Work: The study was retrospective and cross-sectional, limiting causal inference regarding temporal accumulation. Future work requires longitudinal, micro-randomized field trials to validate these findings in real-world, sustained interactions.