How Fast Do Signatures Learn? Statistical Theory and Applications for Path Regression

Problem Statement

Many modern prediction and decision-making problems in operations research involve path-valued covariates—data that evolve over time, such as intraday price paths, battery diagnostic pulses, or electroencephalogram (EEG) windows. In these settings, the response variable often depends not merely on the terminal state of a trajectory but on the entire path, including timing, local fluctuations, and ordered interactions among coordinates.

While path signatures (iterated integrals of a path) have become a canonical feature representation for such data, their theoretical justification has historically relied on the universal approximation theorem. This theorem guarantees that a continuous path functional can be approximated by a linear functional of a signature. However, this is a qualitative existence result; it does not quantify how fast the approximation error decreases as the truncation level $K$ increases. Without a quantitative rate, it is difficult to determine the optimal truncation level or to establish statistical consistency for regression estimators that rely on truncated signatures.

Methodology

The paper develops a unified framework for signature-based path regression by bridging approximation theory with statistical learning. The methodology proceeds in three stages:

1. Approximation Theory: The authors derive an $L^2$ approximation rate for truncated signatures applied to smooth functionals of Itô diffusions. They define a class of smooth coefficient functionals generated by time and Stratonovich integrals. By analyzing the relationship between signature coordinates and polynomial approximations of Wiener–Itô chaos kernels, they establish that the squared approximation error decays at a rate of $O(K^{-2\gamma})$, where $\gamma$ characterizes the smoothness of the functional. They further prove this rate is minimax optimal.
2. Statistical Learning Frameworks: The authors propagate this approximation rate through three standard statistical learning procedures:
   • Signature-OLS: Ordinary Least Squares regression using truncated signature features.
   • Signature-LASSO: Least Absolute Shrinkage and Selection Operator for sparse recovery in high-dimensional signature dictionaries.
   • Signature-Logistic: Logistic regression for binary classification of path-valued data.
     For each, they establish consistency and asymptotic properties by explicitly balancing the estimation error (which grows with the number of features $d_K$) and the approximation error (which decays with $K$).
3. Empirical Validation: The theoretical findings are validated through Monte Carlo simulations and three real-world applications:
   • Finance: Forecasting foreign exchange realized volatility from intraday price paths.
   • Energy: Predicting battery end-of-life (EOL) from early diagnostic current-voltage pulse paths.
   • Medicine: Detecting epileptic seizures from short EEG windows.

Key Contributions

1. Quantitative Approximation Rates
The paper provides the first explicit $L^2$ approximation rate for truncated signatures applied to Itô diffusions.

• Result: For a class of smooth functionals with smoothness parameter $\gamma$, the squared approximation error is of order $K^{-2\gamma}$.
• Significance: This transforms the qualitative universal approximation theorem into a quantitative tool, showing that smoother functionals admit faster convergence. The authors also demonstrate that this rate is minimax sharp.

2. Statistical Consistency with Truncation Error
Unlike previous works that treat truncated signature models as exact, this paper explicitly models the truncation residual.

• Signature-OLS: Establishes consistency and asymptotic normality. The analysis shows that for polynomial decay rates, the truncation level $K$ must grow logarithmically with the sample size $n$ (specifically $K \sim \log n$) to ensure the approximation error does not dominate the estimation error, given the exponential growth of the full signature dimension.
• Signature-LASSO: Proves support recovery and coefficient estimation consistency under sparsity assumptions. The results show that even with a large dictionary, if the active set of features is small, consistent recovery is possible provided the approximation rate is sufficiently fast.
• Signature-Logistic: Establishes consistency for the latent logit score in binary classification, extending the approximation-estimation tradeoff analysis to non-linear link functions.

3. Empirical Framework and Applications
The paper demonstrates that signatures provide informative finite-dimensional representations that often outperform handcrafted features, provided the learning procedure is matched to the statistical regime (e.g., using regularization for high-dimensional signatures).

• FX Volatility: Signature-OLS outperformed standard HAR-family benchmarks in forecasting next-day realized volatility across 12 currency pairs.
• Battery EOL: Signature-LASSO significantly reduced prediction error compared to handcrafted features and unregularized OLS, highlighting the need for sparse regularization in high-dimensional signature settings.
• Seizure Detection: Signature-Logistic achieved superior balanced accuracy and AUC compared to handcrafted logistic regression and random forest baselines, demonstrating the ability to capture ordered waveform interactions.

Results

• Theoretical: The derived rate $O(K^{-2\gamma})$ is shown to be minimax optimal. The paper clarifies that without stronger smoothness (e.g., analyticity leading to exponential decay), one cannot use the full signature dictionary; instead, one must rely on feature selection (LASSO) or logarithmic truncation.
• Simulation: Numerical experiments confirmed the $K^{-2\gamma}$ decay rate, the approximation-estimation tradeoff (where intermediate $K$ yields best finite-sample performance), and the ability of LASSO to recover sparse structures in signature expansions.
• Real Data:
  • In FX forecasting, Signature-OLS reduced RMSE by an average of 7.9% relative to the best non-signature baseline.
  • In battery prediction, Signature-LASSO improved test $R^2$ by 0.430 over the best handcrafted baseline, whereas unregularized Signature-OLS failed due to overfitting.
  • In seizure detection, the signature classifier achieved a balanced accuracy of 95.8%, outperforming both handcrafted logistic regression and random forest models, particularly in sensitivity.

Significance and Claims

The paper claims to provide a quantitative foundation for signature-based path regression. Its primary significance lies in moving beyond the "existence" of signature approximations to a rigorous understanding of convergence rates and statistical consistency.

The authors emphasize that while signatures are powerful, their practical success depends on:

1. Smoothness: The rate of approximation is governed by the smoothness of the target functional.
2. Regularization: In high-dimensional settings (where $d_K$ grows exponentially with $K$), regularization (LASSO, Ridge) or feature selection is essential to prevent overfitting and ensure stable finite-sample performance.
3. Tradeoff: The choice of truncation level $K$ must balance the bias introduced by truncation against the variance introduced by estimating a large number of parameters.

The paper concludes that the proposed framework offers a unified approach for using signatures in path regression, applicable to dense continuous-response regression, sparse high-dimensional regression, and binary classification. It notes that future work could extend these results to non-smooth functionals (e.g., barriers, stopping times) and dependent samples.