💻 computer science

Arabic Clinical Decision Support under a Limited GPU Budget: LoRA versus Full Fine-Tuning for Medical Question Answering

This study demonstrates that while full fine-tuning initially outperforms default low-rank adaptation (LoRA) for Arabic clinical question answering, a well-configured LoRA strategy—particularly 4-bit QLoRA—can effectively match full fine-tuning performance under limited GPU budgets, rendering it an adequate and efficient adaptation choice.

Original authors: Ibrahim Abaker Hashem, Omar Elgendy, Ali Bou Nassif, Ismail Shahin, Ashraf Elnagar, Ayad Turky, Imad Afyouni

Published 2026-09-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ibrahim Abaker Hashem, Omar Elgendy, Ali Bou Nassif, Ismail Shahin, Ashraf Elnagar, Ayad Turky, Imad Afyouni

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the bustling world of modern medicine, doctors rely on vast libraries of knowledge to diagnose illnesses and guide treatments. For decades, this knowledge has been stored in textbooks and journals, mostly written in English. Today, powerful computer programs known as large language models can read these texts and answer questions, acting as a digital assistant that knows more than any single human could memorize. These tools are becoming essential for clinical decision support, helping to triage patients, explain complex conditions, and answer medical queries. However, a significant gap remains: most of these intelligent systems are trained primarily on English data. For the hundreds of millions of people who speak Arabic, the medical information available in their own language is often scarce, and the computer programs that could help them are not yet ready. The challenge is not just about translating words; it is about teaching a computer to understand the specific way medical questions are asked and answered in Arabic, a language with a rich and complex structure that differs greatly from English.

Researchers at the University of Sharjah set out to solve a practical problem facing hospitals and health ministries in Arabic-speaking countries: how to build a reliable medical question-answering assistant when the budget for computer power is tight. These organizations often have access to only a single graphics processing unit, a specialized chip used for heavy calculations, rather than the massive clusters of chips that tech giants use to train the most advanced models. The team needed to know which starting computer model to choose and, more importantly, how to teach it the necessary medical skills without running out of memory or money. They compared two different ways of teaching these models. The first method, called full fine-tuning, involves retraining every single internal setting of the computer program to fit the new medical task. This is like taking a master carpenter and having them relearn every single tool and technique from scratch to build a specific type of chair; it is thorough but requires a massive workshop and a lot of time. The second method, known as low-rank adaptation, is more like giving that same carpenter a specialized, lightweight toolkit that allows them to build the chair quickly without forgetting their original skills. This approach updates only a tiny fraction of the computer's settings, making it possible to run on a single, modest computer chip.

To test these approaches, the researchers selected four different computer models, ranging from general-purpose systems that know a little bit about Arabic to specialized systems built specifically for the language. They then put these models through a two-stage training process. First, they exposed the models to a large collection of real-world Arabic medical conversations, where patients ask questions and doctors provide free-text answers. This was intended to familiarize the computers with the natural flow of medical language. In the second stage, they tested the models on a standardized exam consisting of nearly 4,800 multiple-choice questions covering twenty-five different medical specialties, from emergency medicine to oncology. The goal was to see which combination of starting model and teaching method produced the most accurate answers.

The results offered a clear, if nuanced, path forward for teams working with limited resources. The researchers found that for the two primary models they tested, the thorough method of retraining every setting did perform slightly better than the lightweight toolkit approach, but only when the toolkit was used with its default settings. The difference in performance was small, roughly equivalent to getting one or two extra questions right out of a hundred. More importantly, this small advantage disappeared when the researchers adjusted the settings of the lightweight toolkit or used a specific memory-saving technique called 4-bit quantization. In these optimized cases, the lightweight method matched the performance of the heavy, full retraining while running on a single computer chip. This suggests that for most practical purposes, the expensive, resource-heavy method is not strictly necessary.

The study also revealed a surprising fact about the starting models. The computer models that were originally trained with a heavy focus on Arabic language performed significantly better than the general models when they were first asked to answer questions without any further training. However, once all the models were taught the specific task of answering medical exam questions, this initial advantage vanished. The specialized Arabic models and the general English-focused models ended up with nearly identical scores after training. This indicates that the specific medical task itself is the most important factor in determining success, rather than how much Arabic the model saw before it started learning.

Furthermore, the researchers discovered that the first stage of training, where the models read thousands of free-flowing medical conversations, did not actually help them perform better on the multiple-choice exam. In fact, for some models, this extra step made their final scores slightly worse. It appears that the skill of writing a natural, open-ended medical response does not automatically translate to the skill of selecting the correct answer from a list of options. The format of the task matters deeply; a model trained to chat with patients does not necessarily become better at taking a standardized test.

Ultimately, the study provides a concrete recommendation for health systems operating under financial constraints. They do not need to spend their entire budget on massive computer clusters or on training models for months on general medical text. Instead, they can select a strong, available computer model and use the lightweight, memory-efficient training method on a single graphics card. By focusing their resources on the specific task of answering medical questions rather than on broad, preliminary training, they can achieve high levels of accuracy. The research confirms that a well-configured, efficient approach is sufficient for Arabic clinical decision support, allowing these vital tools to be deployed in hospitals and clinics where they are needed most, without requiring the resources of a large technology corporation.

Technical Summary: Arabic Clinical Decision Support under a Limited GPU Budget

Problem Statement

Teams deploying Arabic-language clinical question-answering (QA) assistants often face severe computational constraints, typically operating with a single 24 GB GPU rather than the multi-GPU clusters required for full-parameter updates. This creates a critical decision point: should they employ Full Fine-Tuning (FFT), which updates all model parameters but requires significant resources (e.g., four 24 GB GPUs for 8B models), or Parameter-Efficient Fine-Tuning (PEFT) methods like Low-Rank Adaptation (LoRA), which fit on a single GPU?

Furthermore, it is unclear how the pre-training language profile of the base model interacts with this choice. Does a model with extensive Arabic pre-training (e.g., Jais-2-8B) require full fine-tuning less than a general-purpose, English-dominant model (e.g., Llama-3.1-8B)? Additionally, the efficacy of a two-stage pipeline—first adapting to a general Arabic medical corpus (AraMed) and then to a specific multiple-choice benchmark (MedAraBench)—remains unexplored in the Arabic medical domain.

Methodology

Experimental Design

The study employs a two-stage fine-tuning pipeline:

Stage 1 (Domain Adaptation): Continued pre-training on AraMed, a corpus of 109,834 open-ended Arabic medical question-answer pairs.
Stage 2 (Task Adaptation): Fine-tuning on MedAraBench, a cleaned dataset of 16,756 Arabic medical multiple-choice questions (MCQs) across 25 specialties.

The authors compare four adaptation strategies across these stages:

LoRA $\rightarrow$ LoRA
LoRA $\rightarrow$ Full Fine-Tuning
Full Fine-Tuning $\rightarrow$ LoRA
Full Fine-Tuning $\rightarrow$ Full Fine-Tuning

Base Models

The core comparison focuses on two 8-billion-parameter models representing opposite ends of the Arabic pre-training spectrum:

Jais-2-8B-Chat: Arabic-centric, with 33% of its pre-training tokens in Arabic.
Llama-3.1-8B: General-purpose, predominantly English pre-training.

Two additional 7–9B models were included for broader validation: Fanar-1-9B-Instruct (Arabic-centric) and Qwen2.5-7B-Instruct (General-purpose).

Configuration and Evaluation

Hardware: Experiments were conducted on AWS g5.12xlarge instances (4x NVIDIA A10G GPUs). Full fine-tuning required all four GPUs; LoRA/QLoRA ran on a single GPU.
LoRA Settings: Default configuration used rank $r=16$ , $\alpha=32$ , and dropout $0.05$, targeting all linear projection layers. A hyperparameter sweep (rank, learning rate, dropout) was also performed.
Evaluation: Models were evaluated on the MedAraBench test set (4,761 samples) using log-probability selection (selecting the token with the highest logit among A–E) rather than autoregressive generation. Primary metrics were Accuracy and Macro-F1.
Reliability: Key configurations were rerun with three random seeds, and results were analyzed using bootstrap confidence intervals and McNemar's tests.

Key Contributions

Two-Stage Pipeline Analysis: This is the first study to systematically separate Arabic medical QA adaptation into domain adaptation (AraMed) and task adaptation (MedAraBench), providing diagnostic evidence on whether open-ended medical text pre-training transfers to structured MCQ reasoning.
Factorial Comparison: The paper provides a rigorous comparison of LoRA vs. Full Fine-Tuning across models with varying degrees of Arabic pre-training, isolating the effect of pre-training language from model size.
Hyperparameter Sensitivity: Beyond default configurations, the study includes a LoRA hyperparameter sweep and 4-bit QLoRA experiments to determine if performance gaps are due to the method itself or suboptimal tuning.
Reliability Analysis: The authors quantify the stability of performance differences, distinguishing between genuine method advantages and seed-level variance.

Results

1. Full Fine-Tuning vs. LoRA (Default Config)

For the two original 8B models (Jais and Llama), Full Fine-Tuning at Stage 2 outperformed default-configured LoRA by 1.3–1.6 Macro-F1 points.

Llama-3.1-8B: Full FT (51.0%) > LoRA (49.4%). This gap was statistically reliable across seeds.
Jais-2-8B: Full FT (51.3%) > LoRA (50.0%). This gap was within seed-level variation and not statistically robust.

However, this advantage disappeared when:

LoRA hyperparameters were tuned (e.g., increasing rank to 64 or setting dropout to 0).
4-bit QLoRA was used, which matched Full Fine-Tuning performance (51.3% for both models) on a single GPU.
The Fanar-1-9B and Qwen2.5-7B models were tested, where default LoRA actually matched or exceeded Full Fine-Tuning.

2. Impact of Pre-training Language

Zero-Shot: Arabic-centric models (Jais) held a significant zero-shot advantage (6.5 Macro-F1 points over Llama).
Post-Fine-Tuning: This advantage largely vanished after task-specific fine-tuning. The final performance of Jais (51.4%) and Llama (51.0%) became nearly identical, suggesting that task-specific adaptation is the dominant factor, regardless of the base model's pre-training language.

3. Domain Adaptation (Stage 1)

The AraMed domain adaptation stage yielded no reliable downstream benefit.

While Stage 1 successfully reduced perplexity on AraMed, it did not improve Macro-F1 on MedAraBench compared to direct Stage-2 fine-tuning.
In some cases, conditioning on a Stage-1 checkpoint slightly degraded performance. The authors attribute this to a task-format mismatch: open-ended consultation text (AraMed) does not transfer effectively to structured MCQ reasoning (MedAraBench).

4. Error Analysis

Knowledge vs. Reasoning: The majority of errors (80%) were attributed to gaps in specific medical knowledge rather than clinical reasoning failures.
Difficulty: Performance dropped significantly on advanced years (Y3–Y5) compared to introductory years (Y1–Y2), indicating that the models struggle with complex, multi-step clinical reasoning.
Data Quality: A non-trivial portion of errors (7%) were linked to annotation errors or ambiguous gold labels in the benchmark itself.

Significance and Claims

The paper concludes that for teams deploying Arabic clinical QA assistants under limited GPU budgets:

LoRA (specifically tuned or 4-bit QLoRA) is an adequate adaptation strategy. It matches Full Fine-Tuning performance when properly configured and fits on a single 24 GB GPU.
Full Fine-Tuning is not a categorical necessity. Its modest advantage over default LoRA is model-dependent, configuration-sensitive, and does not generalize to all model families tested.
Domain Adaptation is not strictly necessary. Spending budget on a separate domain adaptation stage (AraMed) before task tuning does not guarantee better results and may be inefficient compared to direct task fine-tuning.
Pre-training Language is less critical than expected. While Arabic-centric pre-training helps in zero-shot scenarios, it does not fundamentally alter the choice of adaptation strategy or the final performance ceiling after fine-tuning.

Limitations Acknowledged by the Authors:

The study is limited to the 7–9B parameter scale; results may differ for 70B models.
Evaluation is restricted to MCQs; the models' ability to generate safe, open-ended clinical advice is untested.
The performance levels (51–59% Macro-F1) are far from the reliability required for actual clinical deployment, and the authors explicitly state these models are not ready for patient-facing use. The work is presented as an adaptation-strategy study, not a clinical validation.