Insurance Pricing Optimization via Off-Policy Evaluation

Technical Summary: Insurance Pricing Optimization via Off-Policy Evaluation

Problem Formulation
Traditional insurance pricing relies on risk-based principles (e.g., GLMs, GAMs) that ensure actuarial fairness and solvency but often neglect policyholders' price sensitivity and demand dynamics. While dynamic pricing and reinforcement learning (RL) have gained traction in other industries, their application in actuarial science remains limited, often relying on "predict-then-optimize" (PTO) frameworks. These PTO approaches first estimate a demand function from historical data and then optimize prices based on that estimate. However, such methods are susceptible to bias if the demand model is misspecified or if historical pricing policies induce selection effects, leading to systematically suboptimal decisions and inflated revenue estimates.

This paper frames insurance pricing as a stochastic control problem within an off-policy evaluation setting. The objective is to find a pricing policy $\pi$ that maximizes the expected reward (e.g., profit) $V(\pi)$ using historical data collected under potentially different, suboptimal logging policies $\tilde{\pi}$. The framework treats pricing decisions as actions $A$ taken on policyholders with features $X$, generating rewards $R$. A key challenge is that data for any single policyholder is typically available under only one action, necessitating methods to estimate the value of counterfactual policies without new experiments.

Methodology
The paper proposes a four-part methodological framework:

1. Kernelized Inverse Propensity Score (IPS) Estimator:
   The authors introduce a novel estimator that extends the classical IPS method. While classical IPS relies on exact matches between the logging policy's actions and the target policy's actions (discarding non-matching data and suffering from high variance), the proposed kernelized IPS estimator exploits local structure in the action space.

   • Mechanism: It assumes a local regression structure in the action variable (e.g., expected rewards vary smoothly with price). It constructs a kernel matrix $K(x)$ that maps actions from the historical space $\mathcal{A}$ to a new target space $\bar{\mathcal{A}}$ (which may differ in resolution or range).
   • Construction: The kernel is derived via a weighted least squares problem that minimizes the discrepancy between observed rewards and a local regression model (e.g., linear or quadratic in actions).
   • Properties: Under a mild local regression assumption, the estimator is proven to be unbiased. Furthermore, it achieves variance reduction compared to classical IPS by aggregating information across neighboring actions rather than discarding data. The paper characterizes a variance-optimal kernel (using the inverse of the conditional reward covariance matrix) and proposes a computationally efficient "naive" kernel (using diagonal weights) that performs comparably in practice.

2. Policy Optimization Approaches:
   Building on the value estimates, the paper investigates two methods to compute the optimal pricing policy $\pi^*$:

   • Data-Shared Lasso (DSL): An interpretable approach that formulates the policy optimization as a regularized regression problem. It approximates the kernelized IPS estimator by removing explicit dependence on individual rewards, solving for action-specific deviations from a global linear model using Lasso regularization.
   • Neural Network (NN) Parameterization: A flexible, non-parametric approach where a feedforward neural network maps policyholder features to action probabilities (via a softmax output). The network is trained via stochastic gradient ascent to maximize the kernelized IPS estimate directly.

3. Synthetic Environment:
   The methods are evaluated in a controlled synthetic travel insurance environment. The data generation mimics a realistic scenario where customers decide to purchase based on a price-sensitive demand model involving linear and higher-order interactions (including a latent variable for destination). The ground truth for expected rewards is known, allowing for rigorous benchmarking.

Key Results
Empirical experiments on the synthetic data yield the following findings:

• Variance Reduction: The kernelized IPS estimator significantly reduces the Root Mean Square Error (RMSE) compared to the classical IPS estimator across all sample sizes. It achieves the same accuracy as the classical IPS estimator with roughly half the sample size.
• Naive vs. Optimal Kernel: The computationally efficient "naive" kernel (diagonal weights) performs nearly identically to the variance-optimal kernel (requiring covariance estimation) in the tested setting, suggesting practical scalability without complex moment estimation.
• Extrapolation Capability: Unlike classical IPS, the kernelized estimator can evaluate policies on action spaces extending beyond the historical data (e.g., price adjustments of $\pm 30\%$ when data only covers $\pm 20\%$). While accuracy degrades at the extremes due to non-linear truncation effects in the demand model, the method successfully interpolates and moderately extrapolates.
• Policy Optimization Performance:
  • Neural Networks: NN policies consistently achieve the smallest performance gap relative to the theoretical optimum, particularly when the data generation process includes complex, higher-order interactions that linear models miss.
  • Data-Shared Lasso: DSL provides an interpretable linear structure with action-specific deviations, performing competitively with the classical PTO approach but with better robustness to model misspecification in value estimation.
  • Bias in PTO: The study confirms that the classical "predict-then-optimize" approach suffers from selection bias when the same demand model is used for both optimization and value estimation, leading to overestimation of policy performance. The off-policy kernelized IPS estimator avoids this bias.

Significance and Claims
The paper claims to provide a principled, low-risk pathway for data-driven insurance pricing that separates experimentation from deployment. Its primary contributions are:

1. Offline Optimization: Enabling the evaluation of an unlimited number of counterfactual pricing strategies using a single randomized data collection phase, avoiding the high costs and operational constraints of online A/B testing in insurance.
2. Variance Reduction and Extrapolation: The kernelized IPS estimator offers a statistically rigorous method to reduce estimator variance and evaluate policies on new or finer-grained action spaces, addressing a key limitation of standard off-policy methods.
3. Robustness: By decoupling the value estimation (via unbiased off-policy estimators) from the policy optimization step, the framework mitigates the bias inherent in "predict-then-optimize" loops.
4. Practical Applicability: The demonstration that computationally efficient approximations (naive kernels, Lasso) perform well suggests the framework is viable for real-world actuarial applications where interpretability and computational efficiency are paramount.

The authors conclude that while flexible neural networks offer superior performance in capturing complex demand dynamics, the proposed framework provides a robust, unbiased alternative to traditional risk-based pricing, particularly in settings where demand sensitivity is a critical factor.