🤖 machine learning

FMMVCC: Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series

This paper introduces FMMVCC, a Mamba-based deep clustering framework that combines state space sequence modeling with multi-view self-supervised learning to efficiently capture long-range temporal dependencies in univariate time series, achieving state-of-the-art performance across 15 benchmark datasets without requiring manual labels.

Original authors: Donato Cerciello, Leonardo Schiavo, Angel Panizo-LLedot, Javier Huertas Tato, David Camacho

Published 2026-07-09✓ Author reviewed ⓘ

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Donato Cerciello, Leonardo Schiavo, Angel Panizo-LLedot, Javier Huertas Tato, David Camacho

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a librarian trying to organize a massive, chaotic room filled with thousands of different audio recordings. Some are songs, some are bird calls, and some are just static. The problem? You have no labels. You don't know what any of the recordings are, and you can't ask a human to listen to them all and write down descriptions. This is the world of Unsupervised Time Series Clustering: trying to group data based on its patterns without a teacher telling you the answers.

The paper introduces a new tool called FMMVCC to solve this problem. Here is how it works, explained through simple analogies:

1. The Problem with Old Tools

Traditional methods for sorting these "recordings" are like trying to compare two songs by only listening to the first five seconds. They often miss the big picture or get confused by long, complex patterns. Newer, more powerful tools (like Transformers) are like super-computers that can listen to the whole song, but they are so heavy and slow that they struggle when the room gets too big (too much data).

2. The Secret Weapon: "Mamba"

The authors chose a new type of engine called Mamba. Think of Mamba as a highly efficient, selective librarian.

The Old Way: A traditional librarian reads every single word in every book to find a pattern, which takes forever.
The Mamba Way: Mamba is smart. It knows exactly which words matter and which ones are just noise. It can scan a massive library in linear time (meaning if the library doubles in size, the work only doubles, it doesn't explode). This allows it to handle very long sequences of data without getting tired or slow.

3. The Training Method: "The Blindfolded Game"

How does the computer learn to group these recordings without labels? The paper uses a technique called Multi-View Contrastive Learning. Imagine a game of "Telephone" or a detective training exercise:

The Setup: You take one recording (a time series).
The Masking: You create several "views" of it. In some views, you cover up (mask) random parts of the recording with static. In others, you speed it up, slow it down, or add a little bit of background noise.
The Goal: You show these different, slightly broken versions to the AI. The AI's job is to realize, "Hey, even though this version has static and that one is sped up, they are actually the same song!"
The Result: By forcing the AI to see the "soul" of the data through all these different, messy lenses, it learns a very strong, robust understanding of what the data actually is, ignoring the noise and missing pieces.

4. The Fuzzy Sorting Hat

Once the AI understands the data, it needs to sort it into piles (clusters).

Old Methods: These are like a strict judge who says, "This recording belongs only to Group A." But what if a recording sounds a little bit like Group A and a little bit like Group B?
FMMVCC (The Fuzzy Approach): This uses "Fuzzy Clustering." Think of it like a sorting hat that says, "This recording is 70% Group A and 30% Group B." This is crucial because real-world data is often messy and ambiguous. It allows the AI to handle uncertainty gracefully rather than forcing a rigid, wrong answer.

5. The Results: The Winner's Circle

The authors tested this new system on 15 different "rooms" (datasets) containing various types of data, from heartbeats to stock prices.

The Scoreboard: Out of 60 different ways to measure success, FMMVCC came in first place 29 times and had the best average rank across the board.
Why it won: It was better at finding the true structure of the groups (not just guessing based on surface similarities) and was much more stable than other methods.
Efficiency: They also proved that their "Mamba" librarian is much faster and uses less memory than the "Transformer" super-computers, especially when the data gets very long.

Summary

In short, FMMVCC is a smart, efficient system that learns to sort complex, messy data streams by:

Using a fast, selective engine (Mamba) to read long patterns.
Training itself by looking at the same data through many different "broken" lenses (Multi-View Masking) to learn what truly matters.
Using a flexible sorting method (Fuzzy Clustering) that admits when data is ambiguous, rather than forcing a rigid choice.

The paper claims this makes it the current champion for organizing unlabeled time-series data, outperforming both old-school methods and newer, heavier deep learning models.

Technical Summary: FMMVCC – Fuzzy Mamba-based Multi-View Contrastive Clustering for Univariate Time Series

Problem Statement
The proliferation of IoT devices and sensor networks has generated vast volumes of univariate time series data across domains such as healthcare, industrial systems, and smart cities. However, obtaining labeled data in these environments is often expensive or infeasible, necessitating unsupervised learning approaches. While time series clustering is crucial for organizing data into meaningful groups, existing deep clustering methods face significant limitations. Traditional methods (e.g., K-means with DTW) struggle with complex, high-dimensional temporal dynamics. Deep learning approaches, particularly those based on Transformers, often suffer from quadratic computational complexity relative to sequence length, making them inefficient for large-scale or long-sequence data. Furthermore, many current methods fail to robustly handle missing data or capture long-range dependencies efficiently without incurring high computational costs.

Methodology
The paper proposes FMMVCC (Fuzzy Mamba-based Multi-View Contrastive Clustering), a deep clustering framework designed to address these challenges through three core components:

Mamba-Based Representation Learning:
The framework utilizes Mamba encoders, which employ a selective state space model (SSM). Unlike recurrent networks or Transformers, Mamba processes sequences with linear time complexity ( $O(T)$ ) while maintaining the capacity to model long-range temporal dependencies. This architecture allows for efficient processing of long time series without the computational bottlenecks associated with attention mechanisms.
Multi-View Masked Representation Learning:
To enhance robustness against noise and missing sensor data, the method generates multiple views of each time series through stochastic temporal augmentations (jittering, scaling, permutation) and temporal masking. The masking strategy simulates realistic missingness patterns by combining structured (contiguous segments) and unstructured (scattered points) drops. These views are processed independently by dedicated encoders.
Contrastive and Reconstruction Objectives:
The pretraining phase employs a multi-task objective to learn discriminative representations:
- Intra-view Reconstruction: Decoders reconstruct the original input from the latent representation of each view.
- Cross-view Reconstruction: Latent representations from one view are used to reconstruct the latent representations of other views to enforce consistency.
- Contrastive Loss: An instance-level contrastive objective maximizes the similarity between representations of different views of the same sample while contrasting them against other samples in the batch. A mixup-based strategy is used to generate harder negative samples.
Fuzzy Clustering-Oriented Finetuning:
Following pretraining, the model undergoes a finetuning stage to structure the latent space. A temporal pooling mechanism aggregates sequence representations into fixed-length vectors. A fuzzy clustering objective is then applied using a cosine-similarity-based softmax over learnable cluster prototypes. This objective comprises three terms:
- Entropy: Encourages confident cluster assignments.
- Balance: Mitigates cluster collapse by ensuring uniform cluster utilization.
- Separation: Promotes diversity among cluster prototypes to improve inter-cluster distance.

Key Contributions
The authors identify three primary contributions:

Mamba-based Representation Learning: The introduction of a deep clustering framework that leverages Mamba encoders to model long-range dependencies with linear computational complexity, offering an efficient alternative to Transformer and RNN-based models for large-scale sequential data.
Multi-View Masked Representation Learning: A strategy generating diverse views via stochastic augmentations and random temporal masking. This enables the model to learn robust representations invariant to temporal perturbations and missing data, addressing uncertainty often ignored by deterministic enhancement methods.
Cluster-Oriented Representation Learning: The integration of a fuzzy clustering objective into the representation learning process. This approach produces soft cluster assignments, allowing the model to handle ambiguous temporal patterns and improve cluster separability in the latent space.

Experimental Results
The framework was evaluated on 15 univariate time series datasets from the UCR Time Series Archive. The performance was compared against five baselines: TimeSeriesKMeans-DTW, HDBSCAN, FCACC, PG-Mamba, and EMTC.

Performance: FMMVCC achieved the best overall performance in 29 out of 60 total metric evaluations (across F1, NMI, ARI, and RI metrics) and secured the highest average rank (1.85) across all tested scenarios.
Robustness: The model demonstrated superior stability compared to other deep learning baselines like FCACC, which showed higher variability in performance.
Ablation Studies:
- Number of Views: An analysis of the number of views ( $N$ ) revealed that $N=4$ offers the optimal trade-off between representation richness and computational efficiency, with performance degrading at $N=8$ due to redundant noise.
- Loss Components: Sensitivity analysis confirmed that the separation weight ( $\lambda_s$ ) effectively enhances inter-cluster distance, while the balance weight ( $\lambda_b$ ) improves intra-cluster compactness.
- Encoder Directionality: A comparison between unidirectional and bidirectional Mamba encoders showed that the unidirectional (causal) configuration is superior, with the bidirectional variant causing significant drops in ARI (-12.04%) and NMI (-11.57%), likely due to temporal inconsistencies introduced across masked views.
- Scalability: FLOPs analysis confirmed that the Mamba encoder scales linearly with sequence length ( $O(T)$ ), whereas a comparable Transformer encoder exhibited quadratic growth ( $O(T^2)$ ), validating the efficiency of the proposed approach for long sequences.

Significance and Claims
The paper claims that FMMVCC represents a significant advancement in unsupervised time series clustering by effectively combining the linear-time efficiency of State Space Models with the robustness of multi-view contrastive learning. The authors assert that their approach successfully captures long-range temporal dependencies while handling the noise and missing data common in real-world IoT environments. By integrating a fuzzy clustering objective, the framework provides a mechanism to deal with ambiguous temporal patterns that hard clustering methods may struggle with.

The authors conclude that while the framework is currently limited to univariate sequences and lacks direct interpretability of latent representations, it establishes a strong baseline for efficient, scalable, and robust time series clustering. Future work is identified as extending the framework to multivariate time series to capture inter-variable correlations and enhancing the explainability of the learned features.