🤖 machine learning

DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

The paper introduces DynaTab, a deep learning architecture inspired by neural rewiring that dynamically reorders high-dimensional tabular features based on intrinsic complexity and processes them through an order-aware fusion module, achieving statistically significant performance gains over 45 state-of-the-art baselines across 36 diverse real-world datasets.

Original authors: Al Zadid Sultan Bin Habib, Gianfranco Doretto, Donald A. Adjeroh

Published 2026-05-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Al Zadid Sultan Bin Habib, Gianfranco Doretto, Donald A. Adjeroh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, messy box of Lego bricks. Some are red, some are blue, some are tiny, some are huge. In the world of computer science, this box is called "tabular data." Usually, when we feed this box into a smart computer brain (a deep learning model), we just dump the bricks in the order they happened to fall out of the box.

Here's the problem: The computer brain is confused. It doesn't know if it should look at the red bricks first, or the big ones, or the tiny ones. It's like trying to read a book where the words are shuffled randomly on every page. Sometimes the story makes sense; other times, it's gibberish.

Enter DynaTab, a new invention by researchers at West Virginia University and the University of Utah. Think of DynaTab not as a new brain, but as a super-organized librarian who rearranges the books before you even open them.

The "Neural Rewiring" Magic Trick

The secret sauce of DynaTab is inspired by how our own brains work. When you learn a new skill, like playing guitar, your brain doesn't just sit there; it actually rewires itself. It strengthens the connections between the neurons that help you play the song and prunes away the ones that don't.

DynaTab does the same thing with data. Instead of keeping the features (the Lego bricks) in a fixed, random order, it uses a "neural rewiring" algorithm to shuffle them dynamically. It asks: "Which features should be neighbors to tell the best story?"

The Sorting Hat: First, DynaTab checks if the data is even worth sorting. It uses a special math trick called the Intrinsic Dimensionality Factor (IDF). If the data is already simple and organized (like a neat stack of pancakes), DynaTab knows sorting won't help much. But if the data is a chaotic mess of thousands of features (like a pile of 10,000 Lego bricks), the IDF tells DynaTab, "Hey, this is messy! Let's reorder!"
The Rewiring: Once it decides to sort, it groups similar features together (like putting all the red bricks in one pile) and then rearranges them based on how important they are. It's like a conductor telling the orchestra to play the violins before the drums because that's what makes the song sound best.
The Reading: Finally, the reordered data is fed into a standard computer brain (like a Transformer or Mamba model), but now the brain can actually understand the story because the words are in the right order.

What DynaTab Says "No" To

The researchers were very clear about what doesn't work. They argue against the idea that we should just ignore the order of features entirely. Some older models tried to be "order-agnostic," meaning they claimed it didn't matter if the bricks were shuffled. The paper shows that this is a mistake for complex data; the order does matter, and ignoring it leaves performance on the table.

They also found that for very simple, small datasets (like a tiny box with only 5 bricks), this fancy sorting isn't necessary. In those cases, old-school methods like Lasso or simple Decision Trees often do just as well, if not better. DynaTab isn't a magic wand for every problem; it's specifically designed for the "high-dimensional" nightmares where the number of features is huge compared to the number of examples.

The Proof: Did It Actually Work?

The team didn't just guess; they tested DynaTab on 36 different real-world datasets. These ranged from medical records (gene expression data) to image analysis (like recognizing cats vs. dogs).

The Results: In the messy, high-dimensional worlds (where there are way more features than data points), DynaTab was a superstar. It beat 45 other state-of-the-art models.
- On a dataset called GLI-85 (brain tumor data), DynaTab hit 85.96% accuracy, beating the next best model.
- On Dog vs. Cat images, it reached 99.20% accuracy.
- Across all the high-dimensional tests, it consistently ranked first or second, with an average rank of 2.63 (where 1 is the best).
The Limits: The paper is honest about where it stumbles. On the low-dimensional datasets (the "small boxes" with few features), DynaTab didn't win. For example, on the Adult census dataset, it ranked 11th out of the top models. The researchers suggest this is because when data is already simple, the extra work of sorting doesn't add value, and simpler models are faster and just as accurate.

How Sure Are They?

The authors are confident, but they use the right words. They say their results show "statistically significant gains" on high-dimensional data. They ran rigorous statistical tests (like the Friedman test and Wilcoxon-Holm tests) to prove that DynaTab's success wasn't just luck.

However, they don't claim it's a "solved problem." They admit that for very large datasets (over 100,000 samples), the model can get memory-hungry and might need help from other tools (like the Mamba backbone) to run efficiently. They also note that for the simplest datasets, the "rewiring" step is overkill.

The Bottom Line

DynaTab is like a smart librarian who realizes that the order of books on the shelf changes how well you can find the story you're looking for. By dynamically rearranging the data based on its own complexity—mimicking how our brains rewire themselves to learn—it helps computer brains understand messy, high-dimensional data much better than before.

It's not a cure-all for every data problem, but for the really messy, complex puzzles where the pieces seem to have no pattern, DynaTab suggests a new way to play: reorder the pieces first, then solve the puzzle.

Technical Summary of "DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data"

1. Problem Statement

High-dimensional tabular data lacks the inherent spatial or sequential structure found in images and text, posing a significant challenge for deep learning models. Traditional architectures like CNNs and Transformers often underperform on tabular tasks, particularly in High-Dimensional Low-Sample Size (HDLSS) regimes. While tree ensembles remain strong baselines, deep learning approaches struggle with the "Column Permutation Problem" (CPP): the sensitivity of model performance to the arbitrary order of input features. Existing attempts to impose structure, such as fixed feature ordering (TabSeq) or permutation-invariant representations (TabICL), fail to generalize across diverse datasets or adapt to intrinsic data complexity. Furthermore, in HDLSS contexts, simple models often outperform complex deep architectures, suggesting that feature selection and ordering are critical, yet static selection is insufficient when $n \ll m$ .

2. Methodology: DynaTab

The authors propose DynaTab, a deep learning architecture inspired by neural rewiring and neuroplasticity. The core premise is that feature ordering should be a learnable, dynamic process that adapts to the intrinsic complexity of the dataset, mimicking how the brain strengthens or weakens synaptic connections based on stimulus relevance.

Key Components:

Intrinsic Dimensionality Factor (IDF) & Feature Ordering Effectiveness (FOE): Before training, the model quantifies dataset complexity using IDF (the ratio of intrinsic to total feature count). A lightweight criterion, FOE, predicts whether dynamic feature ordering will yield significant gains. High FOE (low IDF) indicates redundancy and high potential benefit from reordering.
Dynamic Feature Ordering (DFO) via Neural Rewiring:
- Clustering: The dataset is clustered (e.g., via K-Means) to form local feature graphs.
- Local Rewiring: Within each cluster, a neural rewiring algorithm optimizes feature ordering. This process mimics synaptic plasticity (Hebbian learning), where edge weights between features are updated based on centrality scores (degree, betweenness, etc.). High-centrality features are prioritized, and weak connections are pruned to minimize intra-cluster dispersion.
- Global Ordering: Local orderings are aggregated into a global ordering ( $O_g$ ) that minimizes weighted global dispersion, ensuring cohesive structure across the entire feature space.
Order-Aware Fusion (DOF): The reordered features are processed through a specialized module before the sequential backbone. This module includes:
- Order-aware Positional Embedding (OPE): Learns dynamic positional embeddings based on the reordered sequence, replacing fixed sinusoidal encodings.
- Positional-Importance Gating Layer (PIGL): Modulates feature contributions based on ordering-derived importance scores.
- Dynamic Masked Attention (DMA): Enforces directional attention flow consistent with the global feature order, preventing reverse attention in attention-based backbones.
Backbone & Loss: The architecture is compatible with various sequential backbones (Transformer, Mamba, LSTM, DAE). It is trained end-to-end using a composite loss function combining standard prediction loss (BCE/CCE/MSE) with DFO loss (penalizing local feature smoothness and global ordering coherence) and dispersion loss.

3. Key Contributions

Predictive Criterion: Introduction of a theoretical framework using IDF and FOE to determine when feature ordering is beneficial, quantifying dataset complexity and ordering sensitivity.
Neural Rewiring Mechanism: A novel Dynamic Feature Ordering (DFO) approach that adaptively reorders columns by simulating structural plasticity and synaptic weight updates, optimizing inter-feature connectivity per dataset.
DynaTab Architecture: An end-to-end model integrating order-aware fusion (OPE, PIGL, DMA) with plug-and-play sequential backbones, trained jointly under DFO and dispersion losses.
Comprehensive Benchmarking: A new taxonomy categorizing tabular datasets into five regimes (HDLSS, HDHSS, LDHSS, LDLSS, Mixed) based on sample size ( $n$ ) and feature dimensionality ( $m$ ). The model was benchmarked against 45 state-of-the-art baselines across 36 real-world datasets.

4. Experimental Results

The authors evaluated DynaTab on 36 datasets spanning five structural regimes:

HDLSS (High Dimension, Low Sample): DynaTab achieved the best average rank (2.63) among 45 models, significantly outperforming classical methods (Lasso, KNN) and deep baselines (TabNet, Transformers). It demonstrated robustness in extreme sample-scarcity conditions where traditional deep models often fail.
HDHSS (High Dimension, High Sample): DynaTab secured the top average rank (2.50) across six image-derived tabular datasets, outperforming both deep learning and gradient-boosting methods.
Mixed Regime: DynaTab ranked first on five of eight tasks, showing superior generalization across heterogeneous domains (clinical, geospatial, proteomic).
Low-Dimensional Regimes (LDHSS/LDLSS): In low-dimensional settings, tree-based ensembles (CatBoost, XGBoost) generally outperformed DynaTab. The authors note that DynaTab's gains are most pronounced in high-dimensional settings, while simpler models suffice for low-dimensional data.
Statistical Significance: Friedman tests and Wilcoxon-Holm post-hoc analyses confirmed statistically significant improvements for DynaTab in high-dimensional regimes compared to baselines.

5. Significance and Claims

The paper positions DynaTab as a compelling new paradigm for high-dimensional tabular deep learning. Its primary significance lies in addressing the "Column Permutation Problem" by treating feature ordering not as a static preprocessing step, but as a dynamic, learnable component inspired by biological neural rewiring.

The authors claim that:

Dynamic feature ordering effectively reduces redundancy and exposes long-range dependencies in high-dimensional data.
The proposed neural rewiring mechanism allows deep models to mimic the brain's adaptability, making them more robust to heterogeneous datasets.
The integration of DFO with sequence-sensitive backbones (like Mamba or Transformers) enables state-of-the-art performance specifically in HDLSS and HDHSS regimes, where previous deep tabular models have struggled.
The work provides a necessary shift from permutation-invariant designs to order-aware, data-adaptive architectures for complex tabular data.

The paper remains modest regarding low-dimensional regimes, acknowledging that classical models often remain superior there, and identifies future work in scaling to larger sample sizes and extending to multi-view and continual learning.