Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice

Technical Summary: Auditing and Fixing Economic Validity in Tabular Foundation Models for Discrete Choice

1. Problem Statement

Tabular Foundation Models (TFMs), such as TabPFN and Mitra, have demonstrated superior predictive accuracy on discrete choice tasks compared to traditional econometric models like the Multinomial Logit (MNL). However, a critical gap exists between predictive performance and economic validity. While TFMs excel at pattern recognition, their predictions often violate fundamental economic logic required for policy evaluation:

• Monotonicity Violations: Increasing the price (or cost) of an alternative sometimes increases its predicted demand probability.
• Implausible Willingness-to-Pay (WTP): Derived Value of Time (VOT) estimates are frequently negative or orders of magnitude lower than established benchmarks, implying travelers prefer longer, more expensive trips.
• Availability Non-Compliance: Models assign non-zero probabilities to alternatives that are physically unavailable.

Standard remedies like knowledge distillation (training an MNL on TFM soft labels) fail because the linear utility structure of the MNL cannot capture the non-linear patterns that give TFMs their accuracy advantage, resulting in a "bottleneck" where accuracy is lost without fully fixing the structural flaws.

2. Methodology: The Two-Stage Behavioral Adapter

The authors propose a two-stage adapter that embeds TFM predictions within a utility-maximization framework constrained by economic theory. The model defines the utility $V_k(x_i)$ for alternative $k$ as the sum of a structural component and a foundation model correction:

$$V_k(x_i) = \underbrace{V_k^{struct}(x_i)}_{\text{Economic Structure}} + \underbrace{\alpha \log q_k(x_i) + g_k(q(x_i))}_{\text{FM Correction}}$$

Where:

• $V_k^{struct}$ is a standard utility specification with constrained parameters (e.g., time and cost coefficients constrained to be negative).
• $q_k(x_i)$ is the pre-computed probability from the TFM.
• $\alpha$ is a learned scalar weight.
• $g_k$ is a small neural network mapping the TFM probability vector to a residual correction.

The Two-Stage Training Procedure:
To prevent the TFM from distorting the economic parameters, the model is trained in two distinct stages:

1. Stage 1 (Structural Estimation): The correction term is frozen ($\alpha=0, g_k=0$). Only the structural parameters are trained on the raw data. This is equivalent to fitting a standard MNL, ensuring the coefficients reflect pure economic fundamentals (time/cost sensitivity) without TFM influence.
2. Stage 2 (Correction Learning): The structural parameters are frozen. Only $\alpha$ and $g_k$ are trained to minimize loss using the TFM's predictions as additional explanatory variables.

Economic Guarantees:

• Monotonicity: By parameterizing cost/time coefficients as $\beta = -\exp(\theta)$, the model mathematically guarantees $\beta < 0$. Since the TFM correction is fixed during policy perturbation analysis, higher costs always reduce utility.
• Analytical VOT: The Value of Time is computed analytically as the ratio of structural coefficients ($\beta_{time}/\beta_{cost}$), avoiding numerical differentiation issues.
• Availability: Unavailable alternatives are assigned $V_k = -\infty$, ensuring zero probability.

3. Key Contributions

• Behavioral Audit Framework: The paper establishes a protocol to audit TFMs for monotonicity, VOT plausibility, and availability compliance, revealing that raw TFMs fail these tests significantly (e.g., 40–65% negative VOT estimates).
• Novel Architecture: The introduction of the two-stage adapter that decouples economic structure from predictive accuracy, allowing the model to inherit TFM accuracy gains while strictly adhering to economic theory.
• Empirical Validation: Demonstration that this approach recovers the accuracy gap between TFMs and MNLs without sacrificing structural validity.

4. Experimental Results

The method was evaluated on two transportation datasets: Swissmetro (stated preference) and LPMC (revealed preference from London).

• Swissmetro:

  • Accuracy: The adapter (using either Mitra or TabPFN) achieved 76.6% accuracy, recovering 13 percentage points over the standard MNL (63.7%) and nearly matching the raw TFM (77.7–78.0%).
  • Validity: Achieved 100% monotonicity and 0% availability leak.
  • VOT: Maintained a consistent VOT of 79.7 CHF/hr, aligning with published benchmarks, whereas raw TFMs produced implausible estimates (e.g., 0.30 CHF/hr).

• LPMC:

  • Accuracy: The adapter achieved 71.8–72.1% accuracy, a gain of 2.0–2.3 percentage points over the MNL (69.8%).
  • Validity: Maintained 100% monotonicity and consistent VOTs (1.76 GBP/hr for transit, 18.3 GBP/hr for driving).
  • Robustness: Across 10 subsamples, the adapter consistently showed positive accuracy gains (+2.2 ± 0.5 pp) with perfect monotonicity.

5. Significance and Claims

The paper claims that the adapter provides a practical path for deploying foundation models in policy contexts where economic consistency is non-negotiable.

• Policy Relevance: It addresses the risk of deploying models with inverted price–demand relationships, which could lead to misguided infrastructure investments and fare-setting decisions.
• Generalizability: The mechanism is not limited to MNL; the structural component can be replaced with mixed logit, nested logit, or other constrained choice models, while the two-stage training procedure remains applicable.
• Modest Scope: The authors acknowledge limitations, noting the evaluation is currently restricted to transportation datasets and two TFMs. They state that extending to domains with less settled economic theory (e.g., healthcare) requires domain-specific audit designs. The adapter is designed to degrade gracefully to the standalone MNL if the TFM quality is poor, rather than introducing new errors.