Technical Summary: From Neural Network Decisions to Training Cases

Problem Statement

Neural networks are increasingly deployed in high-stakes domains (e.g., medical diagnosis, credit approval, energy bidding) where decisions require more than feature-level explanations. Auditors and stakeholders need case-level evidence: specific training instances that support a decision, the outcomes those instances carried, and how they shape the current action preference. While feature-attribution methods (e.g., Shapley values) identify influential input variables, they do not explicitly trace decisions back to the training data.

Standard neural networks absorb training experience into learned parameters, making the connection between a deployed decision and its relevant training cases indirect. Although Case-Based Decision Theory (CBDT) formalizes reasoning by aggregating outcome support from remembered cases, applying this framework to standard neural networks with fixed representations remains an open challenge. The core problem is determining when and how the coefficients of a linear readout on a fixed neural representation can be interpreted as CBDT-style similarity weights versus signed geometric influence terms, and how to use this decomposition for post-training auditing without retraining.

Methodology

1. Theoretical Framework: Exact Case Decomposition

The paper establishes an exact mathematical link between Ordinary Least Squares (OLS) readouts on fixed neural representations and case-based reasoning.

• Setup: Consider a trained network with a fixed representation map $\phi(x) \in \mathbb{R}^H$. For a discrete action $a$, the readout is linear: $Q(x, a) = v_a^\top \phi(x)$.
• OLS Readout: The weight vector $v_a$ is fitted via OLS to minimize the squared loss against action-conditional return vectors $r_a$ derived from training cases.
• Exact Decomposition: The authors derive that the readout score for any action $a$ at input $x$ is exactly a weighted sum of training-case returns:
  $$Q(x, a) = \sum_{i=1}^n \alpha_i(x) r_{i,a}$$
  where the coefficient $\alpha_i(x) = \phi(x)^\top G^{-1} \phi(x_i)$ is determined by the empirical Gram matrix $G = \Phi^\top \Phi$ of the training representations.
• Semantics:
  • CBDT Similarity Regime: If the representation is whitened ($G = nI_H$) and coefficients are non-negative, $\alpha_i(x)$ behaves as a standard CBDT similarity weight (scaled inner product).
  • Signed Gram-Geometric Influence: Outside this regime, coefficients are signed and unnormalized. They represent signed Gram-geometric influence, where positive coefficients indicate nearby cases in the representation space that raise the score, and negative coefficients act as far-field corrections that lower or offset the score.

2. Audit Signals

Based on the decomposition, the authors propose three audit signals:

1. Case Audit Score: Decomposes the action score into contributions $\psi_i(a, x) = \alpha_i(x)r_{i,a}$. Sorting by absolute contribution identifies influential cases; sorting by sign distinguishes supportive vs. offsetting cases.
2. Influence Entropy: Measures the coherence of action evidence among the top-$k$ influential cases. Low entropy indicates consistent support for a single action; high entropy suggests conflicting evidence.
3. Case-Influence Risk Diagnostic: Combines influence entropy with a Gini disagreement term among the top-10 cases to flag queries with weak or dispersed support, particularly useful for detecting covariate shifts.

3. Experimental Setup

The method is evaluated on:

• Synthetic CBDT Generator: To verify recovery of known case-preference structures.
• PJM Energy Bidding: Real load data with synthetic price returns for a 3-action bidding problem.
• UCI Datasets: Adult Income (approval task) and Default Credit (risk exposure task) using proxy returns.
• Baselines: Influence Functions, Representer Point Selection, TracIn, and an inner-product baseline.

Key Contributions

1. Exact CBDT-Style Account: The paper proves that for fixed-representation OLS readouts, every action score is an exact aggregation of training-case returns. This provides a direct, non-approximate route from a neural decision to the specific training cases that shape it.
2. Semantic Clarification: The authors characterize the conditions under which case weights function as CBDT similarity weights (whitened, non-negative regime) versus when they must be interpreted as signed Gram-geometric influence. This distinction prevents the conflation of "similarity" with "influence" in audit contexts.
3. Retraining-Free Audit Framework: The proposed audit requires only fitting a top-layer OLS probe on a fixed representation. It does not require access to the original optimization trajectory, retraining the model, or perturbation-based approximations.

Results

• Recovery of Case Structure: In the synthetic CBDT setting, the OLS readout achieved 96.0% test agreement with the optimal CBDT action, and reconstructed support scores showed strong correlation with true CBDT supports across all actions.
• Attribution Quality: Compared to baselines (Influence Functions, Representer Point, TracIn), the proposed method achieved the highest Top-30 action consistency (0.941 vs. 0.909 for Influence Functions) and competitive Pearson correlation with ground-truth case contributions.
• Real-World Performance:
  • PJM: The neural policy and CBDT decision maker agreed on 3221 of 3358 test decisions. Disagreements were concentrated in low-margin load-transition regions. The audit successfully identified that "aggressive" actions had weaker support alignment, flagging them for closer inspection.
  • UCI Tasks: High network-CBDT agreement was observed (0.9408 for Adult Income, 0.875 for Default Credit). The audit successfully identified the "standard approval" action in Default Credit as the least faithfully reconstructed, highlighting ambiguity.
• Diagnostic Utility: The case-influence risk score improved the AUC for detecting incorrect predictions under covariate shifts compared to standard network-margin baselines, demonstrating its utility in identifying weakly supported queries.

Significance and Claims

The paper claims to provide a practical, post-training audit view of neural decisions by making the training-case basis algebraically explicit.

• Modesty on Scope: The authors explicitly state that the exact decomposition applies to fixed-representation OLS readouts. They do not claim that arbitrary end-to-end nonlinear training admits the same decomposition without this reduction.
• Interpretability Boundaries: The framework clarifies that retrieved cases are not uniformly "supportive"; negative coefficients play a crucial role as geometric corrections. The audit is valid to the extent of the probe's fidelity (measured by score reconstruction).
• Practicality: The method offers a trade-off: it incurs higher latency per query (approx. 58.6 ms) compared to sub-millisecond baselines but provides stronger decision-relevant case retrieval without retraining or access to training trajectories.

The work bridges the gap between the theoretical rigor of Case-Based Decision Theory and the practical deployment of neural networks, offering a tool to trace, inspect, and contest high-stakes AI decisions through the lens of their training history.