Fine-Tuning a 7B Advisor on Free-Tier GPUs: An Adapter-Handoff Recipe and a Synthetic-Data Reliability Caution

Technical Summary: Fine-Tuning a 7B Advisor on Free-Tier GPUs

Problem Statement
Fine-tuning large language models (LLMs) for specialized domains, such as international study advising, is often constrained by the resource limitations of practitioners in small institutions or public sectors. While parameter-efficient methods like QLoRA (4-bit quantization + Low-Rank Adaptation) theoretically allow 7B models to be fine-tuned on single 16 GB GPUs, practical deployment faces two specific hurdles:

1. Operational Constraints: Free-tier GPU sessions (e.g., Kaggle, Google Colab) impose hard wall-clock and weekly quotas. A multi-epoch fine-tuning run for a 7B model often exceeds the duration of a single session, preventing completion on a single machine.
2. Evidential Reliability: There is a lack of rigorous evaluation regarding the factual reliability of models fine-tuned on synthetic data. Practitioners often rely on reference-based metrics (e.g., BERTScore) computed against synthetic training data, which may measure fidelity to the generator's distribution rather than ground-truth correctness, potentially masking hallucinations on policy-sensitive topics.

Methodology
The authors propose a two-part approach: an operational recipe for cross-GPU training and a rigorous evaluation framework for synthetic data reliability.

• Cross-GPU Adapter-Handoff Recipe:
  The authors demonstrate a workflow to complete a three-epoch QLoRA fine-tune of Mistral-7B-Instruct-v0.3 across two different free-tier GPUs (a Tesla P100 followed by a Tesla T4).

  • Procedure: Training is paused at the end of a session. Instead of transferring the full optimizer and scheduler state, only the LoRA adapter weights (41.9M parameters, ~100 MB) are saved and loaded onto the second machine.
  • Implementation: The second phase re-initializes the AdamW optimizer and learning rate scheduler. The authors document that this "warm-up" cost is negligible and that the adapter alone is sufficient to continue training productively, keeping peak VRAM usage within the 16 GB budget on both architectures (15.888 GB on P100; 14.741 GB on T4).

• Dataset and Training:

  • Data: A synthetic dataset of 2,676 multi-turn student-advisor conversations was generated using the Gemini 1.0 Pro API. Topics include visas, scholarships, and admissions.
  • Quality Control: The pipeline included schema validation, deduplication, and repetition checks but explicitly did not include a factuality check against authoritative sources.
  • Hyperparameters: LoRA rank 16, $\alpha=32$, 4-bit NF4 quantization, AdamW 8-bit optimizer, and a learning rate of $2 \times 10^{-4}$.

• Evaluation Protocol:
  The authors employ a three-axis evaluation on a held-out set of 50 prompts:

  1. Reference-Based Metrics: SacreBLEU, ROUGE-L, and BERTScore against the synthetic Gemini references.
  2. Blind LLM-as-Judge: A judge from a different model family (Claude Sonnet) scores domain accuracy and helpfulness without knowing the model source.
  3. Source-Verified Factuality Audit: A human-independent check of specific factual claims against authoritative government and university sources.

Key Results
The study yields a counter-intuitive trade-off between distributional fidelity and factual reliability:

• Training Dynamics: The loss decreased from 1.01 to 0.34 over the two phases. The transition between GPUs showed a brief plateau consistent with optimizer re-initialization, followed by stable gradient descent.
• Fidelity vs. Reliability:
  • Reference Metrics: The fine-tuned model showed improved similarity to the synthetic training distribution (BERTScore F1 increased by +0.063).
  • Advising Quality: The blind LLM-as-judge preferred the un-fine-tuned base model on 46% of prompts, compared to only 18% for the fine-tuned model.
  • Factuality: On policy-sensitive topics (healthcare, admissions, scholarships), the fine-tuned model produced four confident factual errors (e.g., incorrect Medicare eligibility rules, non-existent degree programs), whereas the base model produced zero errors on the same prompts.
• Root Cause Analysis: A direct audit of the training data revealed that the four factual errors made by the fine-tuned model were already present in the Gemini-generated training corpus. A random sample audit of the training data indicated that 28–40% of the synthetic answers contained verifiable factual errors.

Significance and Claims
The paper makes two primary contributions, framed with specific modesty regarding their scope:

1. Operational Recipe: The authors provide a verified, reproducible method for completing multi-epoch 7B QLoRA fine-tuning on free-tier hardware by handoffing only the adapter. They clarify that this is a deployment recipe, not a methodological advance in optimization, and that the binding constraint in these environments is session duration, not aggregate compute.
2. Synthetic Data Reliability Caution: The central finding is a cautionary result: fine-tuning on unverified synthetic data can increase a model's fidelity to the training distribution (improving reference-based metrics) while simultaneously degrading factual reliability.
   • The authors explicitly state that the reliability drop is a property of the data pipeline (unverified synthetic generation), not the fine-tuning method or the adapter-handoff procedure.
   • They argue that for policy-sensitive tasks, evaluation must include judgment-independent factuality signals, as reference-based metrics alone can be misleading.

Limitations and Scope
The authors acknowledge that the dataset and evaluation are entirely synthetic, generated by a single model (Gemini 1.0 Pro) and judged by a single LLM judge. The study does not claim generalizability to all domains or languages, nor does it assert that the fine-tuning method is inherently flawed—only that it faithfully inherits the errors of unverified training data. The released artifacts (dataset, adapter, and evaluation harness) are intended for research to reproduce these findings, with explicit warnings against deployment for actual student advising without further factuality gating.