Technical Summary: RACE-Sched

Problem Statement

The Dynamic Flexible Job Shop Scheduling Problem (DFJSP) is a critical online decision-making challenge in modern manufacturing. It requires assigning operations to heterogeneous machines under routing flexibility and precedence constraints while reacting to stochastic disturbances such as machine failures, urgent job arrivals, and processing time variations.

Current approaches face a fundamental trade-off:

• Priority Dispatching Rules (PDRs) are interpretable and fast but lack the flexibility to adapt when bottlenecks shift.
• Deep Reinforcement Learning (DRL) captures complex dynamics but often lacks interpretability and struggles with generalization across different problem scales.
• Large Language Models (LLMs) offer advanced reasoning and code generation capabilities (Code-as-Policy), but their inference latency (seconds) is incompatible with the millisecond-level decision cycles required by industrial control systems.

Existing hybrid frameworks often separate learning from execution, relying on offline training that cannot adapt to unseen disturbances in real-time, or they require synchronous interactions that disrupt the production line.

Methodology: RACE-Sched

To resolve the conflict between real-time constraints and long-horizon reasoning, the authors propose RACE-Sched, an asynchronous agent-based framework. It decouples policy execution from logical reasoning using a dual-stream architecture:

1. Asynchronous Dual-Stream Architecture

• Reactive Stream (Execution): Operates within the critical control loop. It executes a low-latency, compiled symbolic priority rule (Python function) to make dispatching decisions. This stream is deterministic and operates in sub-millisecond time, ensuring it never waits for the reasoning process.
• Deliberative Stream (Reasoning): Operates off the critical path. It leverages an LLM to analyze summary statistics of recent decisions, generate candidate Python heuristics, and validate them in a sandbox. This stream runs on a slower timescale (seconds to minutes) and does not block the production line.

2. The Agentic Loop (Safety-Constrained Evolution)

The Deliberative Stream functions through a four-stage cycle:

• Strategic Planner: Summarizes the current system state (e.g., bottleneck identification) to formulate a high-level directive.
• Syntactic Enforcer: Converts the directive into executable Python code. It enforces safety constraints, limits code complexity, and ensures the code compiles within a restricted execution context.
• Simulation Validator: Evaluates candidate rules in a sandbox environment using a representative pool of instances. It compares the candidate against the incumbent rule using statistical tests (e.g., t-tests) and predefined improvement thresholds.
• Semantic Reflector: Synthesizes diagnostic feedback from the sandbox results to justify acceptance or rejection, logging the reasoning for future iterations.

3. Trigger and Deployment Protocol

Updates are triggered either periodically or when performance metrics degrade relative to a sliding window. Crucially, deployment occurs via an atomic pointer swap. Only rules that pass the sandbox validation are promoted to the active rule set, ensuring the control loop remains safe and responsive.

4. Semantic Knowledge Repository

To enhance transferability and reduce "cold start" inefficiencies, the system maintains an indexed repository of validated heuristics. Each entry includes the policy code, performance metrics, and lightweight metadata (e.g., number of jobs/machines). When a new problem instance arises, the system retrieves similar rules to warm-start the candidate generation process, reducing token consumption and improving robustness across scales.

Key Contributions

1. Asynchronous Symbolic Evolution Framework: A novel architecture that decouples real-time dispatching from LLM-driven heuristic synthesis, ensuring the control loop remains responsive while enabling continuous policy evolution.
2. Safe Deployment Mechanism: A sandbox-based validation protocol where candidate heuristics are rigorously tested offline. Updates are deployed only via atomic pointer swaps if they meet strict acceptance criteria, preventing unverified rules from disrupting operations.
3. Semantic Rule Repository: A mechanism for lifelong learning that indexes validated heuristics with instance metadata, enabling retrieval-based initialization and improving transferability across different benchmarks and problem scales.
4. Comprehensive Evaluation: Extensive testing on GEN-Bench, MK-Bench, and JMS-Bench, including stress tests for machine failures, demonstrating superior performance over DRL and direct LLM baselines.

Experimental Results

• Performance: On GEN-Bench (Normal scale), the RACE-Sched variant using Qwen3-8B achieved a Relative Percent Deviation (RPD) of 5.33%, significantly outperforming the leading DRL baseline (IDDQN at 9.82%) and other LLM-based methods. It consistently achieved lower RPD and better average ranks across all benchmarks.
• Efficiency: The framework reduced input token consumption by approximately 60% compared to direct LLM baselines by utilizing the repository for warm starts and shifting from per-step action generation to compact policy synthesis.
• Latency: The Reactive Stream maintained sub-millisecond latency (e.g., ~0.05–0.09 ms), satisfying industrial real-time constraints. In contrast, the Deliberative Stream operated with latencies of 77–114 seconds per update, a gap mitigated by the asynchronous design.
• Resilience: In machine failure stress tests (JMS-Bench), RACE-Sched demonstrated superior adaptability, restoring throughput to pre-failure levels and achieving the minimum final makespan (29.6) compared to baselines that suffered from static heuristics or high variance.
• Ablation: Removing the repository ("w/o Repo") increased RPD significantly (e.g., from 10.41% to 26.19% on GEN-Small), confirming the value of knowledge retrieval. Removing the iterative loop ("w/o Loop") also degraded performance on complex tasks, validating the need for recursive refinement.

Significance and Claims

The paper claims that RACE-Sched successfully harmonizes real-time constraints with long-horizon reasoning. By confining slow LLM deliberation, sandbox validation, and policy improvement to the background, the framework allows the control logic to improve continuously without impeding the real-time responsiveness of physical machines.

The authors position this approach as a solution to the "frequency mismatch" between cognitive cycles (LLMs) and physical cycles (manufacturing). They assert that their method achieves superior solution quality and robust adaptation to dynamic events compared to state-of-the-art DRL and direct LLM control, while maintaining the interpretability and safety required for industrial deployment.

Limitations Acknowledged:
The authors note that the quality of rule evolution depends on the background LLM's capability; weaker models may generate invalid candidates that are rejected. Additionally, constructing representative sandbox scenarios requires engineering effort, and reaction to entirely new disturbance types may still be delayed if multiple deliberation cycles are needed.