Zero-shot adaptation to order book dynamics

Technical Summary: Zero-shot Adaptation to Order Book Dynamics

Problem Statement
Classical Avellaneda–Stoikov (AS) market-making models offer a fast, structurally robust framework based on a small set of interpretable parameters (volatility, order arrival intensities, inventory risk aversion, and time horizon). However, these models typically rely on a fixed objective function defined in advance. If market regimes shift or trading objectives change (e.g., desired inventory targets, risk appetite, or adverse-selection penalties), the model requires manual reinterpretation or recalibration. The paper addresses the need for an adaptive mechanism that can adjust the trading objective in real-time based on market conditions without retraining the entire model or abandoning the analytical structure of the AS framework.

Methodology
The proposed architecture separates market dynamics from the trading objective, utilizing a "successor measure-style" adaptation mechanism inspired by zero-shot reinforcement learning (specifically Forward-Backward representations). The system operates through three primary components:

1. Market State Estimation ($\xi_t$):
   The current limit order book state ($L_t$) is mapped to a low-dimensional vector of Avellaneda–Stoikov parameters $\xi_t = (\sigma_t, A^b_t, A^a_t, \kappa^b_t, \kappa^a_t, c^b_t, c^a_t)$. These parameters govern the local market dynamics, including volatility, order arrival intensities, and local adverse-selection costs. Unlike classical models where these are fixed, here they are inferred dynamically from the order book.

2. Adaptive Objective Estimation ($z_t$):
   Instead of a fixed reward function, the model estimates a low-dimensional objective vector $z_t$ from a rolling window of recent realized rewards ($W_t$). The reward is modeled as a linear functional of outcome features: $r = z^\top \phi(s, a, s')$, where $\phi$ includes profit/loss, next inventory, squared inventory, and adverse-selection costs.
   The objective vector is estimated using a backward representation $B_\psi$ that maps past state-action pairs into the reward-coordinate space. The estimate is computed as a weighted expectation of realized rewards:
   $$\hat{z}_t = \mathbb{E}_{\rho_t} [r^H(s, a) B_\psi(s, a)]$$
   This allows the model to infer the current market's "reward direction" (e.g., whether it is paying for inventory accumulation or penalizing adverse selection) directly from data.

3. HJB Forward Representation and Scalarization:
   The core value object is a vector-valued Hamilton-Jacobi-Bellman (HJB) forward representation, $F_{HJB}(q, \tau, a, z; \xi)$, which represents the expected future reward features under optimal policies for a given objective $z$.
   The scalar action-value is obtained by projecting this vector onto the current adaptive objective:
   $$Q_{z_t}(s, a) = F_{HJB}(s, a, z_t; \xi_t)^\top z_t$$
   The policy $\pi_{z_t}$ is derived by maximizing this scalar value. In the closed-form implementation, this results in quoting rules similar to classical AS but where the continuation value $h_t(q)$ is computed specifically for the inferred objective $z_t$:
   $$\delta^{b,\star}_t = \frac{1}{\kappa^b_t} - \frac{h_{z_t}(q_t + v) - h_{z_t}(q_t)}{v} + \frac{c^b_t}{v}$$
   (and similarly for the ask side).

Key Contributions

• Adaptive AS Framework: The paper introduces a method to retain the analytical speed and interpretability of the Avellaneda–Stoikov model while replacing the fixed objective with a data-driven, adaptive vector.
• Separation of Dynamics and Goals: It demonstrates that market dynamics (parameters $\xi_t$) and trading goals (vector $z_t$) can be decoupled. The dynamics are inferred from the order book, while the goals are inferred from realized rewards.
• Zero-Shot Adaptation: The approach allows the model to adapt to changing regimes and objectives without retraining, effectively functioning as a zero-shot learner for market-making objectives.
• Closed-Form Solution: The authors derive a closed-form quoting rule that integrates the adaptive objective into the standard AS structure, ensuring the model remains computationally efficient.

Experimental Results
The authors evaluated the proposed "Rolling FB-AS HJB" strategy against two baselines on a reconstructed BTCUSDT limit-order-book dataset:

1. A fixed-latency Guéant–Lehalle–Fernandez-Tapia (GLFT) strategy.
2. A fixed-latency Avellaneda–Stoikov strategy.

Findings:

• Performance: Both baseline strategies (GLFT grid and AS grid) produced negative returns and high drawdowns on the evaluation window. In contrast, the Rolling FB-AS HJB strategy achieved positive returns (0.002167) and a significantly higher Sharpe ratio (28.57 vs. negative values for baselines).
• Risk Management: The adaptive strategy exhibited a substantially lower maximum drawdown (0.000947) compared to the baselines (~0.0027–0.0028).
• Trading Frequency: The adaptive strategy traded significantly less frequently (477.85 daily trades) than the baselines (~4500–4600 daily trades), suggesting the adaptive objective and finite-horizon HJB selection act as a filter against over-trading in noisy conditions.

Significance and Claims
The paper claims to provide an "interpretable bridge between analytical market making and representation-based reinforcement learning." By keeping the Avellaneda–Stoikov dynamics as the physical core and adapting only a low-dimensional objective vector, the framework avoids the "black box" nature of deep reinforcement learning while gaining the flexibility of zero-shot adaptation.

The authors are modest in their claims, noting that the results are from a "controlled simulation study" using reconstructed data and approximate fill attribution rather than native exchange data. They acknowledge that a production implementation would require native event data, stronger calibration of fill probabilities, and more detailed cost modeling. However, the study demonstrates that under identical backtest conditions, the adaptive quoting rule behaves fundamentally differently and more robustly than fixed-objective baselines.