From Baselines to Preferences: A Comparative Study of LoRA/QLoRA and Preference Optimization for Mental Health Text Classification

1. Problem Statement

Mental health text classification is a high-impact NLP application critical for early risk identification and prioritization in digital care pathways. However, the field lacks practical guidance on which optimization strategy to employ. Current literature offers heterogeneous modeling routes (classical trees, encoders, parameter-efficient fine-tuning, and preference optimization) often reported in isolated experimental contexts with different tuning depths. This makes it difficult to distinguish whether performance gains are structural (due to the model architecture) or artifacts of specific setup choices (e.g., data imbalance, objective formulation, or hyperparameter tuning).

The paper addresses the need for a systematic, controlled comparative study to determine:

• How performance changes with objective formulation, adapter choice, and optimizer behavior.
• Whether preference-based optimization (DPO, ORPO, KTO) offers robust gains over supervised fine-tuning (SFT) in mental health contexts.
• How class imbalance affects these different optimization pathways.

2. Methodology

The study employs a unified experimental protocol across three progressive stages, moving from simple baselines to specialized adaptation strategies. The evaluation uses the DAIC-WOZ dataset (Wizard-of-Oz clinical interviews) with derived targets based on the PHQ-4 framework (combining PHQ-2 for depression and GAD-2 for anxiety).

Experimental Stages

1. Baseline Establishment:

   • Classical: XGBoost trained on sparse text features.
   • Encoders: BERT-family models (bert-base-uncased, distilbert-base-uncased, roberta-base).
   • Conditions: Tested under "Vanilla" (standard training) and "Optimized" (temperature-scaled inference) profiles across various context windows (94–512 tokens).

2. Parameter-Efficient Supervised Fine-Tuning (SFT):

   • Methods: LoRA and QLoRA.
   • Variables:
     • Objectives: Discriminative (direct classification) vs. Generative (constrained text generation).
     • Schemas: Label only, Label confidence, Label rationale.
     • Optimizers: AdamW, Adafactor, Adam8bit.
   • Goal: To isolate the effects of objective, schema, and optimizer on stability and peak performance.

3. Preference Optimization:

   • Methods: Direct Preference Optimization (DPO), Odds Ratio Preference Optimization (ORPO), and Kahneman–Tversky Optimization (KTO).
   • Profiles: "Vanilla" training vs. "Rebalanced" training (explicitly adjusting for class imbalance).
   • Backbones: Evaluated with both LoRA and QLoRA adapters.

Evaluation Metrics

• Primary: Macro-F1, Weighted-F1, Macro-Precision, Macro-Recall.
• Secondary: Derived binary targets for Anxiety and Depression to test transfer consistency.
• Reporting: Both "Best-run" (peak performance) and "Mean" (stability across runs) scores are reported to distinguish isolated peaks from robust patterns.

3. Key Contributions

• Unified Optimization Narrative: Instead of introducing a new dataset, the paper provides a reproducible framework for comparing optimization strategies, isolating the impact of architecture vs. tuning vs. data balance.
• Granular Sensitivity Analysis: The study decouples the effects of objective formulation (generative vs. discriminative), output schema, optimizer choice, and context windowing, which are often conflated in standard benchmarks.
• Preference Optimization Insights: It demonstrates that preference optimization is not a monolithic "upgrade" but is highly method-dependent. It identifies ORPO as the superior preference method and highlights that class rebalancing is a decisive factor for DPO and ORPO but ineffective for KTO in this domain.
• Practical Guidelines: Offers a clear decision tree for practitioners: start with transparent baselines, apply controlled tuning, and deploy preference optimization selectively only where gains are demonstrable and consistent.

4. Key Results

A. Baselines and Encoders

• Strongest Performer: bert-base-uncased with temperature-scaled inference at a 384-token window achieved the best baseline score (Macro-F1: 0.3518).
• Competitiveness: XGBoost remained competitive (Macro-F1: 0.3375), outperforming many DistilBERT and RoBERTa configurations, proving that robust boosted-tree modeling is still viable.
• Window Sensitivity: Performance was highly sensitive to context window size; bert-base peaked at mid-to-large windows, while distilbert degraded significantly at larger windows.

B. SFT (LoRA/QLoRA)

• Objective Dominance: Generative objectives consistently outperformed discriminative objectives in both peak and average performance.
  • Best Generative: Vanilla QLoRA + Adam8bit (Macro-F1: 0.3399).
  • Best Discriminative: Optimized QLoRA + AdamW (Macro-F1: 0.3072).
• Schema Impact: "Label only" schemas were the most stable. "Label confidence" showed high variance, and "Label rationale" was the weakest.
• Optimizer Stability: Adafactor provided the most stable mean performance, while Adam8bit produced high peaks but lower means (less stable).

C. Preference Optimization

• Method Heterogeneity: There was a massive spread in performance between methods.
  • ORPO: The clear winner. Under rebalanced training, it achieved the highest overall score (Macro-F1: 0.3798), surpassing all SFT and baseline models.
  • DPO: Showed significant sensitivity to class rebalancing (improving from 0.2474 to 0.3493 with rebalancing).
  • KTO: Consistently the weakest method (Macro-F1 ~0.12) and showed no improvement with rebalancing, suggesting fundamental limitations for this task.
• Rebalancing Impact: Class rebalancing was a high-impact intervention for ORPO and DPO but irrelevant for KTO.

D. Secondary Outcomes (Anxiety & Depression)

• The ranking patterns observed in the primary 4-class task held true for the derived binary anxiety and depression tasks.
• Optimized BERT remained the strongest encoder baseline for secondary tasks.
• ORPO (Rebalanced) maintained its lead among preference methods, confirming that the gains are transferable across related clinical endpoints.

5. Significance and Conclusion

This paper shifts the focus from "which model is best" to "how should we optimize the model." The central conclusion is that optimization effects are highly method-dependent.

• For Practitioners: Do not assume preference optimization is a universal upgrade. If using preference learning, ORPO is the most reliable choice, and class rebalancing is essential for its success.
• For Stability: Generative SFT with LoRA/QLoRA and Adafactor offers a robust middle ground with stable performance.
• For Baselines: Well-tuned encoder baselines (BERT) remain highly competitive and should be the starting point for any new optimization strategy.

The study provides a reproducible, evidence-based framework for selecting training strategies in mental health NLP, emphasizing that gains come from controlled tuning and data balance interventions rather than simply switching model families.