The Time Value of Evolution

Technical Summary: The Time Value of Evolution

Problem Statement
In evolutionary search, particularly within semantic program mutation (e.g., automated trading policy discovery), standard algorithms often treat the fitness of a newly generated child as the immediate return for the mutation that produced it. This "immediate-return" control is blind to delayed utility. A mutation that temporarily lowers fitness (a "weak child") may be a valuable ancestor that opens a productive region of the search space, enabling high-fitness descendants later in the search process. However, immediate-return optimization penalizes such mutations, potentially discarding paths that could lead to superior solutions before the search budget expires. This paper formalizes this dynamic as the Time Value of Evolution (TVE), distinguishing between the value of an immediate offspring and the finite-horizon value of reachable lineages.

Methodology: Lineage-Value Policy Gradients (LVPG)
The authors propose Lineage-Value Policy Gradients (LVPG), a long-horizon actor-critic framework designed to optimize automated trading policies within a finite-budget Markov Decision Process (MDP).

• Evolutionary MDP Formulation: The search is modeled as an MDP with a horizon $H=8$. The state includes the current program, execution metrics, and remaining budget. The action space consists of three behaviorally calibrated mutation radii: Refine, Interpolate, and Explore.
• Core Architecture:
  • Frozen Backbone: An offline-trained Evolutionary Language Model (ELM), based on Qwen3-8B, serves as the mutation operator. It is frozen during the search process to ensure consistent content distribution and compilation capabilities.
  • Adaptive Controllers: Separate Actor and Critic LoRA adapters are trained via Proximal Policy Optimization (PPO).
    • The Actor dynamically selects the mutation intensity (radius) based on the current state and remaining budget.
    • The Critic estimates the finite-horizon lineage potential. It is bootstrapped using multi-step mutation trees (depth-five ternary trees) to evaluate the expected best-so-far improvement over the remaining horizon.
• Credit Assignment Mechanism:
  • PPO-Path (Proposed): Uses a return signal based on the path best-so-far improvement ($G_{path}$). An action receives credit if a later descendant establishes a new path best, even if the immediate child was worse. The discount factor $\gamma = 1$ ensures improvements are not discounted solely due to temporal delay.
  • PPO-Immediate (Baseline): Uses a clipped one-step fitness change ($\gamma = 0$), treating the immediate offspring's fitness as the sole return.
• Training Protocol: The system is evaluated on automated trading policy discovery for S&P 500 E-mini, Silver, and 30-Year Treasury futures. The protocol involves 90 paired runs per condition, utilizing rolling-origin folds, sealed test windows, and strict matching of operators, seeds, and budgets to isolate the effect of the credit horizon.

Key Contributions

1. Conceptual: The paper formalizes the Time Value of Evolution, defining it as the difference between the finite-horizon lineage value and the immediate offspring value. It argues that immediate fitness is an incomplete measure of an action's value in finite-budget search.
2. Methodological: The introduction of LVPG, which decouples search control into specialized policy heads over a shared generative backbone. It integrates state- and budget-dependent mutation control with a mutation-tree-bootstrapped critic to assign credit over realized evolutionary paths rather than single steps.
3. Empirical: Through a matched paired protocol, the study demonstrates that long-horizon credit assignment significantly improves search efficiency and final policy performance compared to immediate-return optimization.

Results
Across 90 paired runs under matched conditions, LVPG (PPO-Path) demonstrated superior performance over immediate-return baselines (PPO-Immediate) and other control strategies:

• Search Efficiency: PPO-Path increased the validation best-so-far Area Under the Curve (AUC) by 0.394 Sharpe units compared to PPO-Immediate.
• Final Performance: The mean sealed-test Sharpe ratio improved from 0.862 (PPO-Immediate) to 1.321 (PPO-Path).
• Search Dynamics: PPO-Path produced fewer temporary regressions (54.2% vs. 59.8%) and recovered from them more frequently (48.0% vs. 39.9%). When regressions occurred, they were deeper but recovered sooner, yielding larger post-recovery improvements.
• Horizon Sensitivity: Increasing the credit horizon from one step to eight steps significantly improved validation AUC (0.599 units), suggesting that multi-step credit is crucial for navigating non-monotonic search landscapes.
• Critic Validation: The critic achieved a Spearman correlation of 0.597 with realized path returns on held-out states, acting as an informative ranker of future lineage potential.

Significance and Claims
The paper claims that finite-horizon lineage value yields more selective non-monotonic search and stronger policies within identical resource constraints. The primary significance lies in demonstrating that treating mutation decisions as investments—valuing them by the future search opportunities they unlock rather than just their immediate offspring—can overcome the myopia of standard evolutionary algorithms.

The authors emphasize that this improvement is achieved without changing the mutation operator or the underlying task objective, but solely by altering the temporal scope of the credit assignment. The results suggest a general principle for finite-budget optimization: immediate offspring fitness is insufficient for evaluating actions in generative program search, and preserving the "time value" of evolutionary paths leads to better policy discovery. The study establishes these gains under controlled historical evaluation, noting that while it does not guarantee live profitability, it provides a robust blueprint for improving generative search within fixed computational budgets.