📊 statistics

Embedded Polygon Symbolic Transfer Entropy (EPSTE): A Geometric Token and Deep Learning Approach to Estimating Transfer Entropy in Neuroimaging Time Series

This paper introduces Embedded Polygon Symbolic Transfer Entropy (EPSTE), a novel framework that enhances the estimation of directed information flow in neuroimaging time series by transforming local temporal morphology into geometric symbolic tokens and utilizing a recurrent neural network with attention-based multiple-instance learning to achieve stable, high-accuracy recovery of ground-truth directed dependencies.

Original authors: David Alexander Finnigan

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: David Alexander Finnigan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Listening to a Noisy Orchestra

Imagine you are trying to figure out who is leading a jazz band. You have a recording of the music (the brain data), and you want to know: Does the drummer influence the saxophone player, or is it the other way around?

In neuroscience, this is called finding directed information flow. Scientists use a mathematical tool called Transfer Entropy (TE) to answer this. Think of TE as a "predictive ear." It asks: "If I know what the drummer did in the last few seconds, can I predict what the saxophone player will do next, better than if I only listened to the saxophone player's own history?"

The Catch: Real brain data (from EEG or MEG machines) is messy. It's like trying to hear that jazz band while standing next to a roaring waterfall. The data is full of noise, it changes constantly, and there isn't enough of it to get a perfect answer. When scientists try to calculate TE directly from the raw sound waves (the "amplitude"), the math gets overwhelmed. It's like trying to count every single grain of sand on a beach to find a pattern; there are too many grains, and the noise drowns out the signal.

The Old Way: Counting Grains of Sand

Previous methods tried to simplify the data by turning the sound waves into simple "up" or "down" arrows (symbols).

The Analogy: Imagine you are describing a painting. The old method says, "If the paint is darker than the average, write 'D'. If it's lighter, write 'L'."
The Problem: This is too simple. A jagged, angry red line and a smooth, gentle red line might both get the same "D" label, even though they look very different. You lose the shape and feeling of the music. This makes it hard for computers to learn the true connections between brain areas.

The New Solution: EPSTE (The Geometric Translator)

The author, David Finnigan, proposes a new method called EPSTE. Instead of just looking at how loud or quiet the signal is, EPSTE looks at the shape of the wave.

1. Turning Waves into Triangles
Imagine you take three consecutive points on the sound wave and connect them. They form a tiny triangle.

The Analogy: Instead of just saying "the music got louder," EPSTE says, "The music formed a sharp, steep triangle," or "The music formed a wide, flat triangle."
Why it helps: A triangle has three properties: how big it is (area), how steep the sides are (angle), and how high it sits (amplitude). By turning the wave into a sequence of these geometric shapes, the computer gets a much richer "vocabulary" to describe the brain's activity. It's like switching from a vocabulary of just "Up/Down" to a vocabulary of "Sharp Spike," "Gentle Hill," and "Deep Valley."

2. The "Bag of Tricks" (Aggregation)
Even with these better shapes, looking at just one tiny triangle (one split second of time) is still too noisy to be sure.

The Analogy: Imagine trying to guess the weather by looking at a single raindrop. You can't tell if it's a storm or a drizzle. But if you look at a whole bucket of raindrops collected over an hour, the pattern becomes clear.
How EPSTE works: The computer looks at thousands of these tiny triangles (windows of time) and groups them together. It uses a smart AI (a Recurrent Neural Network) to look at the whole "bag" of triangles and say, "Okay, looking at all these shapes together, the drummer is definitely influencing the saxophone."

The Results: A Clearer Picture

The paper tested this new method against the old "Up/Down" method using real brain data from people under the influence of psychedelics (MEG scans).

The Old Method: When the computer looked at the whole picture (aggregated data), it was okay at guessing the connections, but it was fuzzy. It missed the strength of the influence.
The New Method (EPSTE): The computer using the geometric triangles was incredibly accurate. It recovered the "ground truth" (the known connections) with near-perfect precision (99% correlation).
The Verdict: The paper concludes that how you describe the data matters more than the computer's power. By translating the messy brain waves into a structured "language of shapes" (triangles), the AI could learn the connections much faster and more accurately.

Summary in a Nutshell

The Goal: Figure out which part of the brain talks to which other part.
The Problem: Raw brain data is too noisy and complex for standard math to handle easily.
The Innovation: Instead of measuring the raw volume of the signal, the new method (EPSTE) measures the shape of the signal by turning it into a sequence of geometric triangles.
The Outcome: This "shape language" allows the AI to see through the noise and accurately map the brain's communication network, proving that describing data in the right geometric way is the key to unlocking brain secrets.

Important Note: The paper is strictly about improving the mathematical method of measuring these connections. It does not claim to diagnose diseases, predict behavior, or change how doctors treat patients yet. It simply shows a better way to do the math.

Technical Summary: Embedded Polygon Symbolic Transfer Entropy (EPSTE)

1. Problem Statement

Inferring directed interactions between neural systems from EEG and MEG data remains a significant challenge due to noise, nonstationarity, and the high sample complexity required by information-theoretic estimators. While Transfer Entropy (TE) offers a principled, model-free measure of directed information flow, its practical estimation is unstable in finite data regimes, particularly as embedding dimensions increase. Classical estimators relying on raw signal amplitudes suffer from the "curse of dimensionality," requiring exponentially large sample sizes to reliably estimate high-dimensional conditional probability distributions.

Existing symbolic formulations of TE (e.g., Symbolic Transfer Entropy) improve robustness by discretizing signals but often rely on limited symbolic alphabets (e.g., ordinal patterns) that constrain the expressive capacity of the state space, potentially discarding critical morphological information such as curvature and magnitude changes. Furthermore, recent deep learning approaches attempting to approximate TE often operate directly on raw amplitudes or implicit distributions, requiring large datasets and careful optimization to remain stable. There is a need for a representation that reduces estimation complexity while preserving the spatiotemporal and structural information central to TE.

2. Methodology

2.1 The EPSTE Framework

The paper introduces Embedded Polygon Symbolic Transfer Entropy (EPSTE), a framework that reframes TE estimation as a learnable problem operating on structured symbolic representations of local temporal morphology rather than raw signal amplitudes.

Geometric Decomposition: Neural time series are decomposed into sequences of geometric primitives derived from local triplets of samples $(x_{t-1}, x_t, x_{t+1})$ $(x_{t - 1}, x_{t}, x_{t + 1})$ . These triplets form triangular primitives encoding three complementary aspects of waveform structure:
1. Triangle Area: Representing the magnitude of change.
2. Angle/Orientation: Derived from relative slopes, representing directional change.
3. Amplitude Change: Representing relative magnitude.
Symbolization: These continuous geometric features are discretized into categorical tokens using quantile-based global bins learned from the training set. The three bin indices are packed into a single symbolic token, creating a compact but expressive state space.
Neural Architecture: A Recurrent Neural Network (RNN) utilizing Gated Recurrent Units (GRUs) is employed to model local temporal dependencies within sliding windows of the transformed symbolic sequences.
Multiple-Instance Learning (MIL): The model operates on "bags" of symbolic temporal windows. An attention-based pooling mechanism aggregates window-level representations into a single trial-level prediction, allowing the network to focus on the most informative segments of the trial.
Training Targets: The network is trained to predict surrogate-validated TE values. Ground-truth labels are generated by computing standard Symbolic TE (STE) from the geometric primitives using a joint-count estimator with a Miller–Madow bias correction.

2.2 Experimental Design

Data: Source-reconstructed MEG data from five participants (psychedelic study), parcellated using the AAL90 atlas (90 regions). Data were preprocessed with notch and low-pass filtering, downsampling to 250 Hz, and z-scoring.
Baseline Comparison: A control condition was implemented using a standard symbolic baseline where raw MEG amplitudes were discretized into categorical symbols using identical quantile binning rules. Crucially, the neural architecture, hyperparameters, optimization procedure, and data splits were held constant between the EPSTE (triangle-based) and baseline (amplitude-based) models to isolate the effect of the input representation.
Evaluation: Performance was assessed at three hierarchical scales:
1. Window-level: Local dynamics.
2. Bag-level (Trial): Aggregation across trials.
3. Pair-level: Aggregation across all trials for directed channel pairs (the primary outcome).
Validation: Surrogate testing (phase-randomization) was used to verify that TE estimates reflected genuine directed structure rather than spurious correlations.

3. Key Results

3.1 Aggregation is Essential

At the window level, both EPSTE and the baseline exhibited high variance and weak correspondence with ground-truth TE values (EPSTE $r \approx 0.48$ , Baseline $r \approx 0.52$ ). This confirms that local windows do not encode stable directed dependence in isolation.

However, aggregation across trials and channel pairs yielded stable directed dependencies:

Bag-level (Trial): EPSTE achieved a strong correlation ( $r = 0.86$ , $R^2 = 0.74$ ) with ground truth, significantly outperforming the baseline ( $r = 0.68$ , $R^2 = 0.46$ ).
Pair-level (Primary Outcome): EPSTE achieved near-perfect recovery of ground-truth directed structure ( $r \approx 0.99$ , $R^2 \approx 0.98$ ). The baseline also improved with aggregation ( $r \approx 0.98$ ) but exhibited higher absolute error and systematic compression toward the mean.

3.2 Quantitative Superiority of EPSTE

Error Reduction: EPSTE achieved significantly lower absolute error than the baseline at the pair level (Mean Absolute Error: EPSTE $\approx 0.0018$ vs. Baseline $\approx 0.0034$ ).
Statistical Significance: A paired Wilcoxon signed-rank test confirmed that the reduction in absolute error achieved by EPSTE was highly significant ( $p \approx 2.9 \times 10^{-15}$ ).
Structural Fidelity: Heatmaps of pair-level predictions showed that EPSTE preserved coherent, graded patterns of directed influence across the network, whereas the baseline produced compressed maps with reduced contrast between strong and weak interactions.

3.3 Learning Dynamics

The EPSTE model converged more rapidly and to a substantially lower loss regime than the baseline. While the baseline plateaued early with a higher loss floor, EPSTE continued to extract informative structure, suggesting that the geometric representation facilitates more efficient learning of directed dependencies.

4. Key Contributions

Novel Representation: Introduction of EPSTE, a framework that decomposes time series into geometric primitives (triangles) encoding magnitude, curvature, and direction, which are then discretized into symbolic tokens.
Demonstration of Representational Impact: Empirical evidence showing that representational structure is a critical factor in the learnability of information-theoretic dependencies. The study demonstrates that a geometric-symbolic representation outperforms raw amplitude-based symbolic representations in recovering directed causal structure, even when using identical neural architectures and supervision.
Multiscale Inference Framework: Validation that stable causal estimates in noisy neural data emerge only through hierarchical aggregation (from windows to trials to pairs), supporting the view that causal structure in neural systems is distributed and requires integration across time to be reliably observed.

5. Significance and Claims

The paper claims that representational geometry plays a critical role in enabling the practical learnability of information-theoretic dependencies.

Beyond Estimators: The findings challenge the assumption that neural models can overcome representation limitations solely through capacity. Instead, the study posits that "representation precedes learning": the ability of neural estimators to recover directed information flow is fundamentally constrained by how temporal structure is encoded.
Balancing Variance and Expressiveness: EPSTE offers a principled compromise between representational richness and computational tractability. By distributing information across multiple interpretable geometric dimensions (area, angle, amplitude) within a single symbolic unit, the method increases state separability without requiring higher-dimensional temporal embeddings, thereby mitigating the curse of dimensionality.
Methodological Shift: The paper argues that causal inference in neuroimaging should not be treated as an estimator-only problem. The choice of representation is a first-order methodological decision that conditions what causal structure can be learned. In data-limited, noisy settings, causal inference is constrained not only by estimators but by the "language" (representation) in which temporal dynamics are described.

Limitations Acknowledged:
The authors note that the study relies on surrogate-based supervision, which provides an operational definition of TE rather than mechanistic ground truth. Additionally, the specific discretization (binning) was fixed rather than learned, and the temporal lag range was constrained to physiologically plausible values. The study does not claim to recover mechanistic causality but rather informational or predictive causal structure.