Technical Summary: Fine-tuned LLM-based Code Migration Framework

Problem Statement

Migrating large-scale database management systems (DBMS) from Oracle to PostgreSQL is a complex, interdisciplinary challenge that extends far beyond simple syntax conversion. The authors identify several critical barriers:

• Semantic and Structural Disparities: Significant differences exist between Oracle PL/SQL and PostgreSQL PL/pgSQL regarding syntax, data types, cursor behavior, and query optimization strategies.
• Scale and Dependency: Large codebases often contain hundreds of thousands of interdependent files, making automated analysis and transformation difficult.
• Limitations of Existing Tools: Traditional rule-based tools (e.g., AWS Schema Conversion Tool, Ora2PG) struggle with deep contextual analysis, dynamic SQL, complex triggers, and exception handling, often resulting in incomplete translations or semantic drift.
• LLM Limitations: While Large Language Models (LLMs) show promise, they frequently suffer from systematic errors such as semantic drift, API call mismatches, and inconsistent control structure mappings when applied directly without domain-specific adaptation.

The core scientific problem is the lack of a unified, iterative framework that balances fine-tuning with prompt engineering to ensure semantic integrity, minimize architectural risks, and achieve high-fidelity migration across massive, heterogeneous code libraries.

Methodology

The proposed solution is an iterative, scalable framework that integrates Generative AI (GAI) with traditional software engineering techniques. The methodology is structured around several key components:

1. Two-Stage Fine-Tuning Rationale

The authors reject a single holistic learning event in favor of a two-stage instruction tuning pipeline:

• Stage 1 (Syntactic Alignment): The model is trained on a dataset pairing code with natural language descriptions (<Code> : <Description>). This phase focuses on learning the structural patterns, dialect nuances, type signatures, and scope rules of both Oracle and PostgreSQL without performing direct conversion.
• Stage 2 (Transformation Behavior): The model is fine-tuned on direct transformation pairs (<Oracle Code> : <PostgreSQL Code>). This stage shifts the model from syntactic understanding to procedural logic conversion, ensuring semantic preservation while transforming blocks, cursors, and exceptions.

2. Hybrid Code Feature Profiling Engine (HCFPE)

To manage the heterogeneity of the codebase, the HCFPE annotates the corpus with SQL feature metadata (syntax markers, flow structures, dialect-specific operators). This converts unstructured code into a structured representation, enabling the creation of stratified datasets (small, medium, large) and ensuring the training data accurately reflects the distribution of features like RMAN, SQL*Plus, and PL/SQL.

3. Knowledge Base Architectures and RAG

The framework employs Retrieval Augmented Generation (RAG) with two distinct strategies:

• Strategy A (Hybrid Knowledge Base): Utilizes three separate vector databases for Oracle code, PostgreSQL documentation, and SME-derived transformation rules. This allows the LLM to triangulate context, applying deterministic rules and authoritative syntax documentation alongside code examples.
• Strategy B (Unified Knowledge Base): Uses a single vector database containing only direct "Oracle-PostgreSQL" transformation pairs. This tests the model's ability to generalize based solely on examples without explicit rule enforcement.

4. Human-in-the-Loop and Error Correction

The framework incorporates a closed-loop iteration logic:

• Evaluation: Outputs are assessed using NLP metrics (Recall, BLEU, ChrF) and structural validators (syntax checkers, static analyzers).
• Error Analysis: Errors are categorized into syntax, semantics, structural, and functional groups.
• SME Feedback: Subject Matter Experts intervene only on ambiguous or complex samples. Their feedback is used to expand the dataset and refine annotations.
• Dataset Estimation: A mathematical mechanism calculates a "Gap" metric ($GAP_{Feature}$) for each SQL feature. This metric combines dataset representation ($GAP_{Dict}$) and model quality ($GAP_{Quality}$) to automatically prioritize which features require additional training samples for the next iteration.

5. Experimental Pipelines

The study evaluates four distinct pipelines:

1. Direct Conversion: Standard prompt-based conversion without external context.
2. History-Aware Conversion: Incorporates previously generated chunks as context to maintain state consistency.
3. RAG Strategy A: Retrieves heterogeneous context (rules, docs, code).
4. RAG Strategy B: Retrieves only paired transformation examples.

Key Results

The study evaluated multiple models (Qwen32B, Qwen7B, GPT-4 variants) against the traditional tool Ora2PG across a dataset of over 1,800 files stratified by size.

• Superiority of Fine-Tuned LLMs: The fine-tuned Qwen32B model (Qwen32B-ft2) consistently outperformed all other candidates and the Ora2PG tool. In the Direct Conversion pipeline, it achieved a file conversion efficiency of 74.57%, compared to 49.66% for Ora2PG.
• Error Reduction: Fine-tuning significantly reduced Syntax Error Rates (SER). The Qwen32B-ft2 model produced the fewest error-containing files (205) and the lowest total error count (3,554) in the conversion pipeline, whereas base models and smaller LLMs accumulated significantly higher error volumes.
• Feature Coverage: Fine-tuned models demonstrated superior alignment with expected PostgreSQL feature distributions. For example, Qwen32B-ft2 achieved 72.3% coverage for CORE_SQL and 69.1% for PL/SQL to PL/pgSQL conversion, whereas Ora2PG showed substantial information loss (e.g., 47.5% PL/SQL coverage) and systematic underproduction of procedural logic.
• RAG Impact: While RAG provided additional benefits, the history-aware pipeline and direct conversion with fine-tuned models yielded the most stable results. Strategy A (Hybrid) offered robustness for complex features, while Strategy B showed strong performance when high-quality paired datasets were available.
• Economic Impact: The authors estimate that a single fine-tuning iteration enabled the automatic migration of approximately 27,690 more files than Ora2PG would have. This translates to a reduction of roughly 9 person-months of manual SME labor for a 100,000-file codebase.

Significance and Claims

The paper claims to provide a methodological foundation for large-scale database migration that addresses the gap between rule-based automation and the semantic complexity of modern codebases.

• Iterative Quality Control: The primary contribution is the "GAP-based dataset estimation" mechanism, which transforms migration from a static engineering task into a data-driven optimization loop. This allows for targeted dataset expansion, ensuring that model improvements are driven by specific performance deficits rather than blind data accumulation.
• Scalability: The framework is designed to handle enterprise-scale libraries (100,000+ files) by decomposing migration quality into feature-level metrics, preventing the exponential growth of manual workload associated with traditional methods.
• Hybrid Approach: The study validates that a combination of two-stage fine-tuning (for structural competence) and RAG (for contextual grounding) yields superior results compared to either approach alone or traditional rule-based tools.
• Economic Viability: By quantifying the reduction in manual effort, the paper argues that LLM-based migration is not only technically feasible but economically advantageous, offering rapid ROI even with modest initial dataset preparation (30k training samples prepared in six sprints).

The authors conclude that while execution-level validation remains a future goal, the proposed framework successfully establishes a scalable, precise, and economically viable approach for enterprise database modernization.