Technical Summary: Two-Stage Prompt Optimization for Few-Shot Relation Extraction

Problem Statement

Automatic Prompt Optimization (APO) remains underexplored for episodic Few-Shot Relation Extraction (FSRE), particularly when utilizing smaller Language Models (LLMs). While prior work has improved FSRE through better reasoning, retrieval, and example selection, the prompts themselves are typically manually engineered rather than automatically optimized. Existing APO methods generally follow two isolated directions: reasoning-based optimization, which uses feedback to make broad semantic and logical improvements in natural language, and gradient-based optimization, which uses loss and gradient signals to identify and edit specific high-impact tokens or spans. The paper posits that these directions are complementary but have not been systematically combined for natural-language prompt optimization in the challenging episodic FSRE setting, where prompts must function alongside a small set of support examples per relation in each episode.

Methodology

The authors propose a two-stage framework that integrates reasoning-based global search with gradient-guided local refinement.

Stage 1: Reasoning-Based Global Search

The first stage employs a reasoning-based optimizer to make broad improvements to the prompt in natural language. The authors evaluate three distinct optimizers in this stage:

• RPO (Reasoning-based Prompt Optimization): An iterative search method that maintains a population of prompts. It samples training examples, collects feedback on successes and failures, and uses an optimizer model to generate revised prompts based on this feedback.
• EvoPrompt-DE: An evolutionary algorithm that maintains a population of prompts and generates new candidates by combining existing ones using a genetic algorithm (specifically the Differential Evolution variant).
• ETGPO (Error Taxonomy-Guided Prompt Optimization): A feedback-driven method that aggregates failures from a large sample, summarizes them into an error taxonomy, and generates a revised prompt in a single step based on the most common error categories.

Stage 2: Gradient-Guided Local Refinement (GradPO)

The second stage introduces GradPO, a novel gradient-guided optimizer designed to refine the output of the first stage. Unlike methods that edit one token at a time, GradPO identifies and edits multiple high-impact spans simultaneously.

• Span Identification: GradPO computes token-level gradients based on the loss over a training subset. It groups nearby high-gradient tokens into editable spans, expanding them until neighboring tokens fall below a gradient threshold.
• Candidate Generation: For each selected span, GradPO generates candidate replacements. This can be done via LLM prompting (GradPO-Gen) or by selecting tokens based on the target model's token probabilities (GradPO-Prob).
• Beam Search: To handle multiple span edits while preserving natural language structure, GradPO uses beam search. It iteratively applies candidate replacements to the current beam of prompts, scoring them using a combined objective of task loss and a perplexity-based fluency penalty.
• Constraints: The method avoids using a separate optimizer model for candidate generation, relying solely on the target model, and disables reasoning during these steps to keep the process lightweight.

Key Contributions

1. Two-Stage Framework: The paper presents the first systematic study of combining reasoning-based global search with gradient-based local refinement for natural-language prompt optimization in episodic FSRE.
2. GradPO: The introduction of a gradient-guided local refiner that edits multiple high-impact spans rather than single tokens, allowing for a richer local search space while maintaining prompt structure.
3. Empirical Validation: Extensive experiments demonstrate that GradPO consistently improves prompts generated by various reasoning-based optimizers (RPO, EvoPrompt, ETGPO) and serves as the most robust second-stage refiner.

Results

Experiments were conducted on FS-TACRED and FS-FewRel using Qwen3-4B and Gemma3-4B as target models.

• Performance Gains: The two-stage framework achieves State-of-the-Art (SOTA) performance on FS-TACRED with Qwen3-4B, improving the F1 score from 27.6 (previous best) to 29.0. On FS-FewRel, the approach remains competitive with prior work.
• Complementarity: The results show that local refinement is highly effective after global search. Across 19 experimental cases, 17 were improved by at least one second-stage method. GradPO was the most consistent refiner, improving 14 of the 19 prompts.
• Stability: While reasoning-based optimization can be unstable across iterations, a single step of GradPO refinement after the first stage provides stable improvements in almost all cases.
• Efficiency: GradPO incurs only a modest additional computational cost (approximately 10.3% of the dev-set evaluation cost in the largest setting) while exploring a significantly richer local edit space compared to single-token baselines like GreaTer.
• Model Specificity: The paper notes that GradPO edits are largely model-specific; cross-model transfer of optimized prompts yields limited gains, as the refinements depend on the target model's specific loss and gradient signals.

Significance and Claims

The paper claims that combining global reasoning-based search with local gradient-based refinement is an effective direction for prompt optimization in episodic FSRE. The authors emphasize that their framework does not require task-specific fine-tuning and relies on smaller LLMs (4B parameters), making it accessible and practical.

The significance lies in demonstrating that:

1. Prompts found by global search often contain locally suboptimal wording choices that can be fixed via gradient-guided refinement.
2. GradPO provides a stable, consistent mechanism for this local refinement, outperforming other baselines like LPO and GreaTer.
3. The approach achieves new performance benchmarks without increasing prompt length, preserving the natural language structure of the instructions.

The authors acknowledge limitations, including the dependency on hyperparameters, the lack of guaranteed improvement for every prompt, and the current focus on non-reasoning tasks and smaller models. They suggest future work could extend this framework to reasoning-based tasks and larger LLMs.