💻 computer science

An Internet Search--Augmented Soft Prompt Tuning Framework for Explainable Resume Screening in Healthcare Recruitment

This study presents an internet search-augmented, soft-prompt-tuned large language model framework that significantly improves the accuracy, explainability, and efficiency of healthcare resume screening by integrating structured extraction, targeted evidence retrieval, and expert rules to prioritize qualified candidates while maintaining human oversight.

Original authors: Yanting Wu, Tianao Hu, Siyu Gu, Yilin Mao, Xingxing Cen

Published 2026-10-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yanting Wu, Tianao Hu, Siyu Gu, Yilin Mao, Xingxing Cen

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Hospitals are vast, complex organizations that rely on a steady stream of skilled people to function, from surgeons and nurses to laboratory technicians and administrative staff. Finding the right person for a specific job is a difficult task known as "person-job fit," where recruiters must weigh a candidate's education, clinical training, research history, and professional licenses against a detailed list of job requirements. In large hospitals, this process involves sifting through thousands of applications, each with a unique format and a different way of presenting the same facts. Traditional computer systems often struggle here because they rely on rigid rules or simple keyword matching, which can miss the subtle context of a candidate's experience. Meanwhile, the newest generation of artificial intelligence, known as large language models, can understand human language with great nuance but often acts like a "black box," making decisions that are hard to explain or verify. The challenge for modern healthcare recruitment is to build a system that combines the language understanding of these advanced models with the transparency and reliability needed for high-stakes hiring decisions.

Researchers at Shanghai Chest Hospital and Shanghai University have developed a new framework designed to solve this problem, creating a tool that helps screen resumes for healthcare jobs while keeping the process clear and auditable. Instead of letting an artificial intelligence model read a resume and immediately spit out a hiring recommendation, the team built a pipeline that breaks the task into smaller, manageable steps. First, the system uses a standard tool to pull specific facts from a resume, such as degrees, work history, and published research, organizing them into a clean, structured format. For candidates applying to research roles, the system then performs a targeted search on the internet to verify the titles of their publications and retrieve the abstracts, ensuring the evidence is real and up-to-date. This verified information is then fed into a large language model, but with a crucial twist: the model's core brain is frozen, meaning its general knowledge remains unchanged, while a small, trainable "soft prompt" is attached to the front. This prompt acts like a specialized instruction set that teaches the model how to evaluate a candidate specifically for the job at hand, without needing to retrain the entire massive system.

The system does not stop at a simple score. It combines the model's nuanced understanding of a candidate's fit with strict, unchangeable rules set by human experts. For example, if a job requires a specific medical license or a certain number of years of experience, the system applies these hard rules automatically. The final ranking is a weighted combination of these hard rules and the model's semantic judgment, producing a list of candidates that recruiters can review. Crucially, for every candidate on the list, the system generates a detailed report showing exactly which pieces of evidence led to their score, which rules were triggered, and how much each factor contributed to the final result. This ensures that the process is not just a guess, but a transparent decision trail that a human recruiter can inspect and understand.

When the researchers tested this framework using real data from 5,419 applications across 38 different positions, the results were significant. They compared their new system against a standard approach where the entire resume and job description are simply pasted into a large language model in one go. The new framework proved far more effective at identifying the right candidates. In tests involving nine different job roles, the system improved the accuracy of its top picks by a substantial margin, correctly identifying qualified candidates in the top review pool far more often than the standard method. For instance, in one role for a thoracic surgery physician, the system's ability to rank the correct candidates jumped from less than 10 percent to over 80 percent. The system also proved highly efficient at narrowing down the pool of applicants; when the recruiters looked at a pool size approximately twice the number of historically selected candidates (2R), the system had already found 97.83 percent of the people who were historically selected by human experts.

The study also revealed which parts of the process mattered most. The researchers found that the module evaluating research and professional experience had the strongest connection to the decisions made by human experts in the past. They also tested whether the internet search for publication abstracts was necessary by removing it from the system. When they did this, the accuracy for research positions dropped noticeably, proving that verifying and reading the actual abstracts of a candidate's work adds real value to the screening process. The system was designed to be a decision-support tool, not a replacement for human judgment. It handles the repetitive work of sorting through thousands of documents and highlighting the most promising candidates, allowing human recruiters to focus their time on the people who truly matter. By combining structured data, verified external evidence, and a transparent scoring method, this framework offers a way to make hospital hiring faster and more consistent without losing the ability to explain why a decision was made. The researchers note that while the results are promising, the system still needs to be tested in other hospitals and in real-time hiring scenarios to confirm its effectiveness across different settings.

Technical Summary: An Internet Search–Augmented Soft Prompt Tuning Framework for Explainable Resume Screening in Healthcare Recruitment

Problem Statement

Hospital recruitment involves a complex "person–job fit" challenge, requiring the evaluation of heterogeneous applicant materials against specific job requirements (e.g., licensure, clinical training, research output). Traditional manual review is time-consuming and prone to inter-reviewer variability, while early automated systems (keyword matching, SVMs, random forests) often lack the flexibility to interpret implicit or context-dependent information and struggle to generalize across different job categories with varying qualification criteria. Furthermore, standard Large Language Model (LLM) applications face two critical issues in this domain: (1) "black box" outputs that are difficult to audit, and (2) reliance on static parametric knowledge that may be outdated or unverifiable, particularly regarding academic publications and professional evidence.

Methodology

The authors propose a hybrid decision architecture that integrates structured evidence extraction, external evidence grounding, and parameter-efficient soft prompt tuning. The framework operates in three main stages:

Evidence Construction and Serialization:
- Parsing: Unstructured resumes (PDF, DOC, TXT) are converted to machine-readable text and parsed into a constrained JSON schema (extracting education, skills, work experience, publications, etc.) using a few-shot hard prompt.
- External Grounding: For research-related positions, publication titles reported in resumes are used as restricted queries to retrieve metadata and abstracts via the Semantic Scholar API. This ensures evidence is traceable and verifiable, avoiding unrestricted web browsing.
- Instance Construction: Candidate–position instances are serialized into a unified text format containing the position description ( $J_i$ ), structured resume ( $R_i$ ), retrieved evidence ( $C_i$ ), and an output instruction, separated by explicit field identifiers.
Soft Prompt Adaptation:
- Architecture: The system utilizes a frozen Qwen backbone. Instead of full-parameter fine-tuning, a compact, trainable soft prompt matrix $P$ (consisting of $m$ virtual-token embeddings) is prepended to the input sequence.
- Training Objective: The model is trained to predict whether a candidate belongs to an archived reference shortlist ( $y_i \in \{0, 1\}$ $y_{i} \in {0, 1}$ ). The loss function combines two components with equal weight:
  - Binary Cross-Entropy ( $L_{CE}$ ): For candidate-level discrimination.
  - Pairwise Ranking Loss ( $L_{rank}$ ): To optimize the relative ordering of reference vs. non-reference candidates within the same position.
- Optimization: Gradients update only the soft prompt parameters ( $\theta$ ), leaving the foundation model parameters ( $\Phi$ ) frozen.
Hybrid Scoring and Ranking:
- Deterministic Rules: Hard criteria (e.g., age, mandatory credentials, journal quartiles) are evaluated using explicit rules. Violations trigger negative penalties.
- Semantic Scoring: The soft prompt-derived matching probability serves as a bounded semantic contribution for context-dependent judgments (e.g., research direction relevance).
- Score Fusion: Final scores are calculated by weighting normalized module scores (Basic Information, Education/Fit, Research/Experience) and adding hard-rule penalties.
- Output: Candidates are ranked by final score to generate a Top-K priority manual-review pool. The system outputs a full audit trail, including extracted evidence, triggered rules, module scores, and weighted contributions.

Key Contributions

Hybrid Decision Architecture: The paper introduces a framework that balances the flexibility of LLMs with the auditability of deterministic rules. It separates evidence extraction, semantic interpretation, and score fusion, ensuring that the final ranking is decomposable and explainable.
Parameter-Efficient Adaptation: By utilizing soft prompt tuning on a frozen Qwen backbone, the framework adapts to specific recruitment tasks with limited labeled data without altering the foundational model's general knowledge.
Restricted External Grounding: The integration of restricted publication retrieval (based on self-reported titles) provides verifiable evidence for research positions, addressing the issue of outdated or hallucinated model knowledge.
Explainable Ranking: Unlike monolithic LLM approaches, this system provides a transparent "audit trail" for every candidate, detailing how specific evidence and module scores contributed to the final ranking.

Results

The framework was evaluated on 5,419 applications across 38 positions at Shanghai Chest Hospital, with detailed ranking experiments conducted on 9 positions (4,339 applications, 150 reference candidates).

Information Extraction: The system achieved an overall F1 score of 98.45% for key-information extraction against manually reviewed standards.
Ranking Performance: Compared to a "Raw Prompt" baseline (single-pass LLM call without modular decomposition), the pipeline-based model significantly improved performance:
- Macro-average R-Precision: Increased from 61.18% to 82.64% (mean improvement of 21.46 percentage points; $P = 0.0078$ ).
- Macro-average NDCG@R: Increased from 0.6632 to 0.8702 ( $P = 0.0039$ ).
- The most significant gains were observed in complex roles like "Thoracic Surgery Physician," where R-Precision jumped from 9.09% to 81.82%.
Review Pool Efficiency: At a manual-review pool size of approximately 2R (twice the number of reference candidates), the system achieved a recall of 97.83%, with diminishing returns observed beyond this point.
Ablation Study: Removing the publication-abstract module reduced the macro-average R-Precision for research positions by 10.32 percentage points, confirming the value of external evidence grounding.
Interpretability: The "Research/Experience" module showed the strongest correlation with archived expert shortlists (Spearman $\rho = 0.587$ ), indicating that the model successfully captured the semantic signals prioritized by human experts.

Significance and Claims

The authors claim that this framework provides a scalable, auditable, and valuable decision-support system for healthcare recruitment. Its primary significance lies in its ability to:

Standardize the extraction of heterogeneous resume evidence.
Prioritize qualified candidates within a manageable review pool, significantly reducing repetitive screening effort.
Expose the underlying evidence and module contributions for every ranking, preserving accountable human oversight.

The paper explicitly states that the system is designed to reduce repetitive screening effort while preserving accountable human oversight, not to replace expert judgment. The authors emphasize that external and prospective validation are required before deployment beyond the study institution and that the system should be viewed as a tool to support, rather than automate, final hiring decisions.