📊 statistics

Area-based epigraph and hypograph indices for functional outlier detection

This paper introduces the Area-Based Epigraph and Hypograph Indices (ABEI and ABHI) to overcome the magnitude-sensitivity limitations of existing methods, proposing the robust EHyOut procedure that integrates these metrics with derivative analysis to effectively detect both magnitude and shape outliers in functional data.

Original authors: Belen Pulido, Alba M. Franco-Pereira, Rosa E. Lillo, Fabian Scheipl

Published 2026-04-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Belen Pulido, Alba M. Franco-Pereira, Rosa E. Lillo, Fabian Scheipl

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a conductor leading an orchestra. Most of the musicians are playing the same song, following the same rhythm, and hitting the same notes. But then, one violinist starts playing a completely different tune, and another trumpet player suddenly blows a note that is so loud it drowns out everyone else.

In the world of Functional Data Analysis (FDA), your "orchestra" is a collection of curves (like temperature charts over a year, population growth lines, or heart rate monitors). The goal is to find the "outliers"—the musicians who aren't playing along.

This paper introduces a new, smarter way to find these musical rebels. Here is the story of how they did it, explained simply.

The Old Way: Counting "Who's on Top?"

Previously, statisticians used a method called the Modified Epigraph Index (MEI).

The Analogy: Imagine a stack of pancakes. The MEI method asks, "How many pancakes are sitting above this specific one?"
The Problem: This method only cares about position, not distance.
- If a curve is just barely above the others for a tiny second, the old method thinks it's a huge outlier.
- If a curve is massively higher than the others (like a skyscraper next to a house) but only for a split second, the old method might miss it because it only counts the time spent above, not how much above.
- It's like judging a race by who crossed the finish line first, ignoring that one runner was actually running on a treadmill while the others were on a track.

The New Solution: Measuring the "Gap"

The authors, led by Belén Pulido, introduced two new tools: ABEI and ABHI (Area-Based Epigraph and Hypograph Indices).

The Analogy: Instead of just counting who is on top, imagine you are painting the space between the curves.
How it works:
- ABEI measures the total area of the space where a curve is above the others.
- ABHI measures the total area where a curve is below the others.
Why it's better: Now, if a curve is a skyscraper, the "paint" (area) needed to fill the gap is huge. The method immediately screams, "That's an outlier!" It captures both shape (the weird curve) and magnitude (the huge jump).

The "EHyOut" Detective: The Three-Step Process

The paper doesn't just stop at measuring the area; it builds a full detective system called EHyOut. Think of it as a three-step investigation:

Look at the Smoothed Line (The Curve): First, they look at the original data (the main melody).
Look at the Changes (The Derivatives): Then, they look at how fast the line is changing (the first derivative) and how that is changing (the second derivative).
- Analogy: If the main curve is a car's speed, the first derivative is the gas pedal (acceleration), and the second is the jerkiness of the ride. Sometimes a car looks normal, but the way it accelerates is weird. EHyOut catches this.
The Multivariate Interrogation: They take all these measurements (Original + Acceleration + Jerkiness) and feed them into a "Comedian Method" (a robust statistical tool).
- Analogy: Instead of asking one question ("Are you weird?"), they ask a panel of experts: "Is the shape weird? Is the speed weird? Is the acceleration weird?" If the answer is "yes" to enough of them, the curve gets flagged as an outlier.

Why This Matters: The Real-World Tests

The authors didn't just talk about theory; they put EHyOut to the test in two real-life scenarios:

Spanish Weather: They analyzed temperature and rain data from 73 weather stations.
- The Result: EHyOut correctly identified that the Canary Islands (which are far away and have a tropical climate) were "outliers" compared to mainland Spain. It also spotted a station in Navacerrada (in the mountains) that was consistently too cold. It found the "weird musicians" in the weather orchestra perfectly.
World Population: They looked at population growth curves for 105 countries.
- The Result: It flagged countries with explosive growth (like many in Africa and the Middle East) and countries with strange, stable patterns (like some in Eastern Europe). It successfully separated the "fast growers" and "slow growers" from the average.

The Bottom Line

Before this paper, finding weird curves was like trying to find a needle in a haystack by only looking at the color of the straw. Sometimes you missed the needle because it was the same color but a different shape.

EHyOut is like a magnet that finds the needle regardless of its shape or size. It is:

Smarter: It measures the "distance" (area), not just the position.
Faster: It runs quickly on computers.
More Reliable: It works whether the outlier is a tiny glitch or a massive explosion, and whether it's a weird shape or just a weird number.

In short, this paper gives statisticians a better pair of glasses to see the "weirdos" in their data, ensuring that the conclusions they draw from the data aren't ruined by a few rogue curves.

1. Problem Statement

Functional Data Analysis (FDA) involves analyzing data where observations are curves or functions. A critical challenge in FDA is outlier detection, as curves can deviate from the majority in multiple ways:

Magnitude outliers: Shifted vertically (amplitude changes).
Shape outliers: Different curvature or form.
Phase outliers: Shifts in the domain (time).
Combinations: Curves exhibiting multiple types of deviations simultaneously.

Existing methods often struggle to detect all types simultaneously. Specifically, the Modified Epigraph Index (MEI) and Modified Hypograph Index (MHI), while effective for shape anomalies, rely on the Lebesgue measure. This means they quantify the proportion of the domain where one curve lies above or below another, but they ignore the magnitude of the vertical distance between curves. Consequently, they fail to flag magnitude outliers or subtle shape deviations where the curves cross frequently but with significant vertical separation.

2. Methodology

The paper proposes a new framework called EHyOut, which integrates novel indices with multivariate outlier detection techniques.

A. New Indices: Area-Based Epigraph and Hypograph Indices (ABEI & ABHI)

The authors introduce two new metrics that quantify the area between curves rather than just the time interval of dominance.

Area-Based Epigraph Index (ABEI): Calculates the total area where sample curves lie above a target curve $x$ .
$ABEI_n(x) = \sum_{i=1}^n \int_I (x_i(t) - x(t))^+ dt$
Area-Based Hypograph Index (ABHI): Calculates the total area where sample curves lie below a target curve $x$ .
$ABHI_n(x) = \sum_{i=1}^n \int_I (x(t) - x_i(t))^+ dt$
Key Advantage: Unlike MEI/MHI, ABEI/ABHI are sensitive to both the duration and the magnitude of deviations. They do not exhibit a linear relationship (unlike MEI/MHI), allowing them to be used jointly as independent features.

B. The EHyOut Algorithm

EHyOut transforms the functional outlier detection problem into a multivariate one through a three-step process:

Data Preparation: The functional data is represented using a cubic spline basis to compute the original curves, their first derivatives, and second derivatives.
Feature Extraction: ABEI and ABHI are computed for the original curves and their derivatives. This results in a feature vector of size $n \times 6$ (Original ABEI/ABHI, 1st Deriv ABEI/ABHI, 2nd Deriv ABEI/ABHI).
Multivariate Detection: The Comedian (COM) method (a robust multivariate outlier detection technique based on the Median Absolute Deviation) is applied to these feature vectors to classify curves as outliers or inliers.

3. Key Contributions

Novel Indices: The introduction of ABEI and ABHI, which overcome the magnitude-insensitivity limitation of traditional epigraph/hypograph indices.
Derivative Integration: The strategy of applying these indices not just to raw data but also to first and second derivatives, capturing anomalies in slope and curvature.
Robust Framework (EHyOut): A unified procedure that does not require prior knowledge of the outlier type (magnitude vs. shape) and performs well across diverse scenarios.
Comprehensive Benchmarking: Extensive evaluation against 19 distinct Data Generation Processes (DGPs) covering various outlier types, comparing EHyOut against state-of-the-art methods like Outliergram, MSPLOT, FASTMUOD, and Functional Boxplots.

4. Results

Simulation Study

The authors evaluated EHyOut against 7 benchmark methods using 19 DGPs with varying outlier proportions ( $\alpha = 0.01, 0.05, 0.1$ ).

Performance Metric: The Matthews Correlation Coefficient (MCC) was used as the primary metric due to its robustness to class imbalance.
Key Findings:
- Consistency: EHyOut was the only method to achieve a median MCC > 0.5 across all 19 DGPs.
- Robustness: While other methods (e.g., Outliergram, MSPLOT) excelled in specific scenarios (e.g., pure shape outliers), they often failed in others (e.g., complex magnitude/shape combinations). EHyOut maintained high performance across all categories.
- Threshold Independence: EHyOut achieved near-perfect AUC (Area Under the Curve) scores, indicating that the method generates excellent ranking scores, even if default thresholds in other methods were suboptimal.
- Efficiency: EHyOut was the second-fastest method (after MBD) with a mean execution time of ~0.008 seconds, making it highly scalable.

Real-World Applications

Spanish Weather Data (AEMET):
- Applied to daily temperature and log-precipitation curves from 73 stations (1980–2009).
- EHyOut successfully identified stations in the Canary Islands as outliers for both temperature and precipitation, consistent with their distinct geographic and climatic patterns. It also flagged mainland stations with anomalous precipitation levels.
UN World Population Data:
- Applied to population growth curves of 105 countries (1950–2010).
- Identified 38 outlier countries, including rapid growers in Africa and the Middle East (e.g., Nigeria, Iraq) and stable/moderate growers in Europe (e.g., Hungary, Czech Republic). The results aligned with known demographic shifts and highlighted countries with consistently high or divergent trends compared to the global median.

5. Significance and Conclusion

This paper addresses a critical gap in Functional Data Analysis by providing a general-purpose outlier detection tool that is sensitive to both magnitude and shape deviations.

Theoretical Impact: It demonstrates that incorporating "area" (magnitude of deviation) into epigraph/hypograph concepts significantly improves detection capabilities over purely topological (time-based) measures.
Practical Impact: EHyOut offers a "plug-and-play" solution for practitioners who do not know the specific nature of outliers in their data. It balances high detection accuracy (MCC) with computational efficiency.
Future Directions: The authors suggest future work on developing adaptive, data-driven thresholds for EHyOut and extending these area-based indices to multivariate functional data settings.

In summary, EHyOut represents a state-of-the-art advancement in functional outlier detection, offering superior robustness and versatility compared to existing benchmarks.