💻 computer science

MSTAR: Multi-Scale Backbone Architecture Search for Timeseries Classification

The paper proposes MSTAR, a novel Neural Architecture Search framework with a multi-scale backbone that simultaneously optimizes frequency and time resolution to automatically discover optimal architectures for time series classification, achieving state-of-the-art performance across diverse datasets and domains.

Original authors: Tue M. Cao, Nhat H. Tran, Hieu H. Pham, Hung T. Nguyen, Le P. Nguyen

Published 2026-08-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tue M. Cao, Nhat H. Tran, Hieu H. Pham, Hung T. Nguyen, Le P. Nguyen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but instead of looking at fingerprints or footprints, you are listening to a long, complicated recording of sounds. This is the world of Time Series Classification. In science, this is the art of looking at data that changes over time—like a heartbeat, the movement of a smartphone, or the flickering of a satellite image—and figuring out what is happening. The data is like a song made of many different notes played at once. Some notes are the main melody (the important information), while others are just static or background noise.

For a long time, scientists trying to solve these puzzles had two main problems. First, they were obsessed with finding the right "notes" (frequencies) to listen to, often ignoring when those notes happened. It's like trying to identify a song by only knowing the pitch of the notes but forgetting the rhythm; you might know it's a rock song, but you can't tell if it's a fast drum solo or a slow ballad. Second, they tried to build one giant, super-complex machine to catch every possible sound at once. This was like trying to build a net with holes so small that it catches every single grain of sand, but the net became so heavy and tangled that it couldn't be dragged anywhere, especially when the beach (the dataset) got huge. The old methods worked okay on small beaches but fell apart when the data got big, or they required humans to guess the perfect design, which is slow and often wrong because every dataset is unique.

Enter MSTAR, a new approach from researchers at Hanoi University of Science and Technology and VinUniversity that acts like a smart, automated architect. Instead of guessing the design or building a clumsy, oversized net, MSTAR uses a technique called Neural Architecture Search (NAS). Think of this as a robot chef that doesn't just follow a recipe but invents new ones. The robot has a massive pantry of ingredients (different sizes of filters and tools) and a magical tasting spoon. It tries out thousands of different combinations of "recipes" (architectures) to find the perfect one for the specific meal (dataset) it is cooking.

The paper suggests that MSTAR's secret sauce is realizing that time resolution is just as important as the frequency. Imagine listening to a heartbeat: you need to know not just that a beat happened, but exactly when it happened to tell if it's healthy or sick. MSTAR searches for a design that keeps the "time" sharp while still catching the right "notes." The researchers found that by letting the computer search for the best structure automatically, they could build models that are much better at handling both tiny datasets (like 10,000 records) and massive ones (like 1 million records) across different fields, from heart monitors to satellite images.

In their experiments, MSTAR didn't just guess; it measured. On a heart dataset called PTB-XL, it found a design that scored a 0.9355 on a scale where higher is better, beating the previous best models. On a satellite image dataset, it improved accuracy to 89.0%, and on a smartphone activity tracker, it reached a score of 0.953. The paper shows that these results weren't just luck; when they removed the parts of the model that kept the "time" sharp, the scores dropped, proving that keeping track of when things happen is crucial. Furthermore, the researchers showed that this new backbone could be paired with powerful "Vision Transformers" (AI models usually used for pictures) to predict eye movements, suggesting that this flexible, time-aware design is a strong foundation for future AI.

The paper argues against the idea that just making the "net" bigger or focusing only on the "notes" is enough. It suggests that the old way of manually designing these models is too slow and often misses the mark because every dataset has its own unique rhythm. Instead, MSTAR suggests that the best way forward is to let an automated system search for the perfect balance of size and timing for each specific problem. While the paper doesn't claim to have solved every mystery in time series, it provides strong evidence that this automated, multi-scale search is a more reliable and scalable way to build the next generation of time-reading AI.

Technical Summary: MSTAR: Multi-Scale Backbone Architecture Search for Timeseries Classification

Problem Statement

Time Series Classification (TSC) faces a fundamental trade-off between capturing diverse frequency components and maintaining time resolution. Previous approaches generally fall into two categories with distinct limitations:

Frequency-Focused Methods: Techniques like Omni-Scale CNN and ensemble methods with various kernel sizes aim to cover a wide range of receptive fields to extract frequencies. However, they often suffer from scalability issues when applied to large datasets. Their fixed or manually designed architectures struggle to adapt to the unique frequency-noise profiles of different datasets, and expanding receptive fields to cover longer signals often leads to overfitting or a significant loss of time resolution.
Deep Learning Models: Models like InceptionTime, 1D-ResNet, and TCN offer better scalability but rely on manually designed structures. These fixed architectures may not capture the optimal set of receptive fields for specific datasets, and the manual design process is time-consuming without guaranteeing optimality.

A critical gap identified by the authors is the overlooking of time resolution. While filtering frequencies is a primary focus in TSC, the authors argue that the time resolution of extracted features is equally critical. Large kernels that aggregate frequencies often blur precise time locations, leading to the loss of crucial predictive information, particularly in non-stationary signals where beneficial frequencies vary across time.

Methodology

MSTAR Framework

The authors propose MSTAR (Multi-Scale Time series ARchitecture search), a Neural Architecture Search (NAS) framework designed to automatically discover the optimal multi-scale convolutional backbone for specific datasets.

1. Multi-Scale Search Space:

Structure: The search space is a cell-based graph inspired by Inception-like architectures (e.g., InceptionTime). It consists of 13 nodes partitioned into three layers, with input and output nodes.
Operations: Edges between nodes represent operations including convolution, max pooling, average pooling, and identity.
Kernel Sizes: The search space allows for flexible kernel sizes. Convolution kernels can be selected from $\{1, 3, 5, 9, 19, 39\}$ , while pooling kernels are chosen from $\{3, 5, 9\}$ .
Channel Management: The design incorporates bottleneck channels (32 channels) and standard channels (128 channels) to minimize computational complexity while allowing for wide receptive fields through stacking.
Goal: This space enables the discovery of architectures that balance frequency extraction with high time resolution, avoiding the excessive parameters of naive multi-scale approaches.

2. Search Strategy:

Algorithm: The framework utilizes Bayesian Optimization (BO) combined with a neural predictor approach, inspired by BANANAS.
Autoencoder (CAE): A Convolutional Autoencoder is pre-trained on the search space to encode the adjacency matrix (representing the architecture) into a continuous vector space. This allows the system to project discrete architectural graphs into a format suitable for regression.
Predictors: A set of predictors is trained to estimate the performance (labels) of architectures based on their encoded vectors.
Optimization Loop:
1. Initialize a population of random architectures.
2. Train predictors on the current population.
3. Use an acquisition function (Expected Improvement) to select promising architectures (via mutation of top performers and random sampling).
4. Evaluate the top candidates by training them from scratch.
5. Update the population and repeat.

3. Integration with Transformers:
The framework is designed to serve as a backbone for Transformer modules. The authors demonstrate pairing the discovered MSTAR backbones with both untrained and pre-trained Vision Transformers (ViT) to leverage global contextual understanding.

Key Contributions

Novel Search Space: Introduction of a multi-scale convolutional backbone search space specifically designed for TSC, capable of finding architectures that optimize both frequency extraction and time resolution.
Automated Architecture Discovery: A framework that eliminates manual design, finding distinct optimal architectures for datasets ranging from small (10K samples) to large (1M samples) across different domains.
Time Resolution Analysis: Analytical experiments demonstrating that time resolution is a critical factor in TSC performance, often overlooked by frequency-only approaches.
Backbone Versatility: Demonstration that the discovered CNN backbones can effectively serve as feature extractors for Transformer models, outperforming standard backbones like InceptionTime when paired with ViT.

Experimental Results

The authors evaluated MSTAR on four datasets across four distinct domains:

PTB-XL (ECG): Achieved an AUC of 0.9355, surpassing the previous state-of-the-art (IncepSE at 0.9343) and other deep learning baselines.
Satellite (SITS): Achieved 89.0% accuracy, outperforming OmniScaleCNN (85.9%) and TCN (86.0%).
Smartphone HAR: Achieved an F1-micro score of 0.953, exceeding the previous state-of-the-art (InnoHAR at 0.945).
EEGEyeNet: When used as a backbone for ViT, MSTAR-VIT-pre-trained achieved an RMSE of 51.3, significantly outperforming the previous state-of-the-art EEGVIT-pre-trained (55.4). Notably, the base MSTAR-VIT model (53.3) also outperformed the pre-trained EEGVIT-pre-trained (55.4) and the standard EEGVIT (61.7).

Ablation Studies:

Time Resolution: Visualizations using Continuous Wavelet Transform (CWT) spectrograms confirmed that larger receptive fields increase frequency intensity but degrade time resolution. Removing nodes with high time resolution (small kernels) significantly degraded model performance, validating the importance of time resolution.
Autoencoder: The Convolutional Autoencoder (CAE) significantly outperformed a Variational Autoencoder (VAE) in reconstructing the search space representation (MSE 5.389 vs. 14.202) and improved the Spearman correlation of the neural predictors.

Significance and Claims

The paper claims that MSTAR addresses the scalability and adaptability limitations of previous TSC methods by automating the search for optimal receptive fields. The authors emphasize that their work shifts the focus from purely frequency-based filtering to a holistic view that includes time resolution.

The significance of the work lies in:

Proving that automated search can yield architectures superior to manually designed ones for specific datasets.
Establishing that time resolution is a critical, often neglected, component of TSC performance.
Providing a scalable backbone that works effectively across diverse data sizes and domains, and can be successfully integrated with modern Transformer architectures.

The authors maintain a modest tone regarding future work, suggesting that while their integration with Transformers is effective, it is a basic implementation and opens the door for more advanced hybrid models. They also note that their search space could be expanded to include operations like dilated convolutions or squeeze-and-excitation modules.