📊 statistics

An `Inverse' Experimental Framework to Estimate Market Efficiency

This paper proposes an "inverse" experimental framework that utilizes quantile-based normalization and machine learning models to predict market allocative efficiency from orderbook data alone, overcoming the limitations of traditional methods that rely on unobservable reservation values.

Original authors: Thomas Asikis, Heinrich Nax

Published 2026-04-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Thomas Asikis, Heinrich Nax

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: The "Reverse Engineering" of Markets

Imagine you are a detective trying to figure out if a crime was committed, but you don't have access to the suspect's diary, their bank statements, or their true motives. You only have the footprints they left behind and the time they were seen at the scene.

In the world of economics, this is exactly the problem researchers face with digital marketplaces (like stock exchanges or online ad auctions).

The Detective's Problem: Economists can see the bids (what people offer to pay) and the asks (what people demand to sell), and they can see the final prices. But they cannot see the true value inside the traders' heads (their "reservation price"—the absolute maximum they are willing to pay or the absolute minimum they are willing to accept).
The Consequence: Without knowing these true values, economists can't tell if the market is actually fair or efficient. They are flying blind.

This paper proposes a "Reverse Engineering" solution.

Instead of the traditional method (where scientists create a fake market with known values to see how it behaves), the authors flip the script. They say: "Let's use a known, controlled market to train a computer, and then teach that computer to guess the hidden values just by looking at the footprints (the bids and asks)."

The Analogy: The Cooking Class vs. The Food Critic

To understand the difference between the old way and this new way, let's use a Cooking Class analogy.

1. The Old Way (The Standard Experiment)

Imagine a famous chef (the economist) gives a student a recipe with exact ingredients (e.g., "Add exactly 200g of sugar"). The student cooks the dish. The chef then tastes it and says, "Did you follow the recipe? Was the sugar level perfect?"

Pros: The chef knows exactly what the "perfect" dish should taste like because they wrote the recipe.
Cons: This only works in the kitchen. In the real world, you never get the recipe. You just get the finished dish.

2. The New Way (The "Inverse" Framework)

Now, imagine the chef trains a Robot in the kitchen.

Step 1: The chef gives the Robot the recipe and the finished dish. The Robot learns: "When I see this specific pattern of stirring and this smell, it means the sugar was perfect."
Step 2: The chef takes the Robot out of the kitchen and puts it in a busy restaurant. The Robot is not given the recipes of the other chefs. It can only smell the food and see the ingredients on the counter.
Step 3: The Robot looks at the ingredients and the smell and says, "I bet this dish has too much sugar," or "This one is perfectly balanced."

This paper is about building that Robot. The authors used experimental data (where they did know the recipes/true values) to train Machine Learning models. Then, they tested if those models could predict market efficiency using only the observable data (bids and asks), just like a real-world analyst would.

How They Did It: The "Quantile" Magic

The data from these markets is messy. It's like trying to predict the weather by looking at a chaotic pile of wind socks, rain gauges, and barometers that are all different sizes and shapes.

The authors realized that feeding raw data into a computer doesn't work well. So, they invented a way to "normalize" the data using Quantiles.

The Analogy: The Race Track
Imagine a race with 100 runners.

Old Way: You list every runner's exact time. If the race is slow, the times are high numbers; if it's fast, they are low. It's hard to compare races.
The Paper's Way: You ignore the exact times. Instead, you ask: "Who is in the top 10%? Who is in the middle (50%)? Who is in the bottom 10%?"
- You create a "Top 10%" line, a "Middle" line, and a "Bottom 10%" line.
- This works whether the runners are running 10 seconds or 100 seconds. It focuses on the shape of the crowd, not the specific numbers.

By turning the messy bids and asks into these "percentile lines," the computer can spot patterns that human eyes miss.

The Results: What Did the Robot Learn?

The authors tested several "Robots" (Machine Learning models) to see which one could best guess if the market was efficient.

The "Gut Feeling" Robot (Linear Models): These are simple models that draw a straight line through the data. They are good at guessing the final price after a few trades have happened, but they struggle at the very beginning.
The "Pattern Master" Robot (Gradient Boosting Trees): This is a complex AI that looks for non-linear, weird patterns.
- The Winner: The "Pattern Master" was the best at predicting efficiency before any trades even happened.
- Why? It could look at the initial "noise" of the bids and asks and say, "Hey, the sellers are being too greedy and the buyers are too scared. This market is going to be inefficient," even before a single deal was made.

Key Finding: The AI could predict how well a market would perform (Allocative Efficiency) with surprising accuracy, sometimes even before a single trade occurred, just by analyzing the "shape" of the order book.

Why Does This Matter? (The Real World)

You might ask, "Why do we care about a computer guessing efficiency in a lab?"

The Answer: Real-World Market Health.

Imagine a massive online marketplace for industrial parts (like semiconductors).

The Problem: The platform owner sees millions of bids and asks. They don't know if the market is fair or if a few big players are manipulating prices. They can't ask every seller, "What is your true cost?"
The Solution: This "Inverse Framework" acts like a Market Health Monitor.
- The system watches the flow of bids and asks.
- It instantly calculates: "This market is running at 60% efficiency. Something is wrong."
- It can alert the platform owner to fix the rules, change the fees, or investigate potential manipulation.

Summary

The Problem: We can't see the true values of buyers and sellers in real markets, so we can't measure if the market is working well.
The Solution: Train an AI on controlled experiments where we do know the values.
The Trick: Teach the AI to ignore the specific numbers and focus on the "shape" of the data (using quantiles).
The Result: The AI can now look at a real market's "footprints" (bids and asks) and accurately predict if the market is efficient, even before a trade happens.

It's like teaching a dog to sniff out a disease by training it on sick patients, so it can then detect the disease in healthy-looking people just by their scent.

1. Problem Statement

Digital marketplaces (e.g., procurement platforms, ad exchanges, B2B commodity markets) generate vast amounts of orderbook data (bids, asks, realized prices). However, assessing the allocative efficiency (how well resources are allocated to their highest-value users) of these markets is fundamentally difficult because:

Unobservability of Valuations: In real-world data, the true reservation prices (maximum willingness to buy, minimum willingness to sell) of traders are private and unobservable.
Limitations of Traditional Theory: Standard economic theory often assumes rational behavior or requires knowledge of these unobservables to calculate efficiency metrics like Competitive Equilibrium (CE) and Gains of Trade (GOT).
Data Complexity: Orderbook data is high-dimensional, non-stationary, non-linear, and lacks a universally accepted model of strategically optimal behavior in continuous double auctions.

The Core Challenge: How can we predict market efficiency and equilibrium prices using only observable orderbook data (bids, asks, prices) without access to the underlying reservation values?

2. Methodology: The "Inverse" Approach

The authors propose an "Inverse Experimental Framework" that reverses the standard logic of experimental economics.

Standard Approach: Induce known reservation values $\rightarrow$ Observe market behavior $\rightarrow$ Test convergence to theoretical equilibrium.
Inverse Approach: Observe market behavior (orderbook data) $\rightarrow$ Predict the underlying market structure and efficiency metrics (which are usually derived from known values).

2.1 Data Source

The models are trained and validated using a large-scale dataset from Ikica et al. [16], a reproduction of Vernon Smith's (1962) continuous double auction experiments.

Scope: 104 markets, ~10 rounds each, with varying sizes (Small/Large), feedback settings (Black Box, Full, Same, Other), and price rules (First, Random, MMK).
Ground Truth: The experimental data includes the induced reservation values, allowing the authors to calculate the true Allocative Efficiency (AE) and Competitive Equilibrium Price (CEP) for validation, even though the prediction models are trained without seeing these values.

2.2 Feature Engineering: Quantile-Based Normalization

To handle the unstructured and variable nature of orderbook data (where the number of active bids/asks changes dynamically), the authors transform raw data into quantile vectors:

Decile Transformation: Active bids and asks are converted into 11 decile vectors ( $D_0$ to $D_{10}$ ), representing the distribution of prices at different probability levels.
Normalization: To ensure scale invariance (generalizing across markets with different price scales, e.g., cents vs. thousands), the authors apply Interquartile Range (IQR) normalization:
$X^\dagger = \frac{X - \text{median}(X)}{\text{IQR}_{[0.35, 0.65]}(X)}$
Input Features: The models use normalized bid/ask quantiles, realized deal prices (if available), round index, number of deals, and categorical market design parameters (feedback setting, price rule).

2.3 Predictive Models

The authors compare four classes of models to predict two targets: Allocative Efficiency (AE) and Competitive Equilibrium Midprice (CEP).

Efficient Market Hypothesis (EMH): A baseline assuming realized prices equal equilibrium prices ( $\hat{p} = p_{realized}$ ) or AE = 1.0.
Corrected EMH (CEMH): A non-parametric estimator using group statistics (median AE/price by market type/round) to correct for initial inefficiencies.
Orderbook Robust Linear Model (OB-RLM): A linear regression on normalized quantiles with robust loss functions (Huber loss) to handle outliers. It includes an intercept to capture scale effects.
Gradient Boosting Trees (GBT): A non-linear ensemble method (using CatBoost/Gradient Boosting) capable of capturing complex, transient market dynamics and interactions between features.

3. Key Contributions

Conceptual Inversion: The first framework to "invert" experimental economics, using controlled lab data to train models that predict efficiency without reservation values, mimicking real-world constraints.
Quantile Representation: A novel method to represent dynamic orderbooks as fixed-dimension quantile vectors, making them compatible with standard machine learning algorithms.
Scale Invariance: A normalization technique allowing models trained on specific price scales to generalize to markets with entirely different price ranges.
Early Prediction Capability: Demonstrating that market efficiency and equilibrium prices can be predicted with reasonable accuracy before any trades occur, solely based on the initial bid/ask distribution.

4. Key Results

The models were evaluated using Median Absolute Percentage Error (Median-APE) on held-out test sets.

4.1 Allocative Efficiency (AE) Prediction

Performance: Non-linear models (GBT) generally outperformed linear models, especially in the first round before any price realization.
- GBT: Achieved Median-APE of ~0.168 (Round 1, 0 deals) and ~0.104 (Round 1, 1+ deals).
- OB-RLM: Performed well after deals occurred but struggled in the very early transient phase (Median-APE ~0.216).
- CEMH: Surprisingly strong in early rounds for small markets but lacked generalizability.
Feature Importance: GBT identified Black Box feedback settings and Round index as critical features. Lower seller ask quantiles were the most predictive of efficiency.
Insight: Linear models catch up as realized prices accumulate, but non-linear models are superior for capturing the initial transient dynamics of market formation.

4.2 Competitive Equilibrium Price (CEP) Prediction

Performance: Linear models (OB-RLM) became highly effective once realized prices were available, often outperforming GBT in later rounds.
- Before Deals: GBT was the clear winner (Median-APE 0.135) compared to OB-RLM (0.191).
- After Deals: OB-RLM achieved the lowest error (Median-APE 0.048), demonstrating that once the market stabilizes, linear relationships dominate.
Feature Importance: For CEP, higher buyer bid quantiles and lower seller ask quantiles were the most influential features.

4.3 Robustness

The models maintained performance across different market sizes (Small vs. Large) and feedback settings.
Ablation studies confirmed that the predictions were not circular (i.e., not simply reusing realized prices) but relied on the structural information in the orderbook.

5. Significance and Applications

Real-World Applicability: The framework offers a tool for platform operators, regulators, and researchers to assess market health and efficiency in real-time without needing private trader data.
Use Cases:
- Supply Chain Procurement: Monitoring efficiency in B2B exchanges for standardized components.
- Ad Exchanges: Detecting inefficiencies or manipulation in real-time bidding.
- Market Design: Testing how different feedback rules or price mechanisms impact allocative outcomes.
Theoretical Shift: The paper argues that when market complexity exceeds analytical tractability, inverse data-driven approaches (Machine Learning) are necessary to extract efficiency signals where traditional forward modeling fails.
Future Work: The authors plan to validate these models on non-experimental, real-world data and extend the framework to other market types (e.g., stock markets).

Conclusion

This paper successfully demonstrates that allocative efficiency and equilibrium prices can be predicted with high accuracy using only observable orderbook data. By leveraging experimental data as "ground truth" to train machine learning models, the authors provide a robust, scalable framework for evaluating digital market performance in environments where the fundamental economic parameters (valuations) remain hidden. The results highlight a trade-off: non-linear models (GBT) are essential for early-stage, transient market analysis, while linear models excel once the market has stabilized and price data is available.