💻 computer science

Mamba-based Selective State Space Modeling Improves the Accuracy-Complexity Tradeoff of SmolVLA Vision-Language-Action Experts

This paper demonstrates that integrating Mamba-based selective state-space modeling into the SmolVLA action expert significantly improves the accuracy-complexity tradeoff for Vision-Language-Action models, achieving superior success rates over Transformer baselines during long-horizon execution while reducing parameter complexity by 24% under per-action replanning.

Original authors: Farida Mohsen, Thowayba Elkaffash, Mohammad Reza Chalak Qazani, Mohamed Mabrok, Nader Meskin, Ali Safa

Published 2026-08-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Farida Mohsen, Thowayba Elkaffash, Mohammad Reza Chalak Qazani, Mohamed Mabrok, Nader Meskin, Ali Safa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Robots are learning to understand the world not just by seeing it, but by reading instructions and then moving their arms to do a job. This field, known as vision-language-action, relies on powerful computer brains that take a picture of a scene and a sentence describing a task, then figure out exactly how to move a robotic hand to complete it. The challenge for engineers is balancing speed with smarts. If a robot thinks too much before every tiny movement, it moves so slowly it becomes useless. But if it plans a long sequence of moves at once to save time, it risks making mistakes because the world might change while it is executing those pre-planned steps. The goal is to find a way for the robot to be both fast and accurate, a balance that has been difficult to strike.

A team of researchers has found a new way to tip this balance in favor of speed without sacrificing accuracy. They tested a new type of computer architecture, originally designed for processing language, inside the brain of a robot. By swapping out the standard thinking engine for this new design, they discovered that the robot could plan much longer sequences of movements and still succeed. In their tests, when the robot was allowed to execute a full sequence of fifty moves before checking its surroundings again, the new design succeeded at the task nearly eight percent more often than the old design. This improvement is significant because it means robots could operate much faster in the real world, making fewer pauses to think, while still getting the job done correctly.

The researchers started with a popular robot brain called SmolVLA, which is already known for being efficient and accurate. This system works by looking at a camera feed and a text instruction, then predicting a "chunk" of future movements. Instead of deciding just one move at a time, it predicts a whole series of actions, like a short movie of what the arm should do next. The robot then carries out the first few moves of that series, stops, looks at the world again, and plans a new series. The number of moves it carries out before stopping is called the execution horizon. If it stops after every single move, it is very accurate but very slow. If it runs the whole series without stopping, it is fast but prone to errors if the environment changes.

To solve this, the team replaced the part of the robot's brain that handles the timing of these movement sequences. The original system used a standard method called self-attention, which is like a spotlight that lets every part of the movement plan look back at every other part to stay coordinated. The researchers swapped this out for a different method called Mamba, which processes information in a way that is more like a steady stream of updates rather than a spotlight. This new method is much lighter on the computer's resources, requiring about twenty-four percent fewer settings to be learned, yet it handles the flow of time differently.

The team put both versions of the robot brain to the test in a simulated environment filled with forty different tasks, ranging from stacking blocks to moving objects to specific goals. They ran the robots through these tasks under three different conditions: stopping after every single move, stopping after twenty-five moves, and stopping after the full fifty moves. When the robots stopped after every move, both versions performed almost identically, with the new design matching the old one's success rate. However, as the robots were asked to run longer sequences without stopping, the difference became clear. The robot with the new Mamba design held its ground much better. When asked to run fifty moves before checking in, the new design succeeded in sixty-one point eight percent of the trials, while the older design only succeeded in fifty-three point nine percent.

This gap grew even wider on specific types of tasks. When the robot had to move objects to precise locations, the new design improved success rates by nearly eighteen percent at the longest horizon. Even on tasks involving spatial arrangements, it improved by ten percent. The researchers found that the new design was particularly good at keeping the later moves in a sequence coordinated with the earlier ones, even when the robot was acting on a plan that was no longer being updated with fresh visual information. This suggests that the new architecture has a natural advantage in predicting how a sequence of actions should unfold over time.

The study confirms that the way a robot plans its movements matters just as much as how often it checks its surroundings. By changing the internal engine that coordinates a sequence of actions, the researchers showed that robots can be made faster and more efficient without becoming clumsy. The new design not only improved the success rate for long sequences but also reduced the total number of computer settings needed to run the system. This means that in the future, robots could be deployed in real-world settings where they need to act quickly and continuously, relying on longer plans that stay accurate for longer periods. The work demonstrates that the right architectural choice can unlock a better tradeoff between speed and precision, bringing the dream of fast, reliable robotic assistants one step closer to reality.

Technical Summary: Mamba-based Selective State Space Modeling for VLA Experts

1. Problem Statement

Vision-Language-Action (VLA) models face a fundamental tradeoff between task success rates and computational latency, governed by the execution horizon ( $N$ ).

High Frequency ( $N=1$ ): Executing a single action per inference ensures the policy remains close to current observations, maintaining high accuracy. However, this incurs significant compute overheads due to frequent policy calls, making real-time deployment infeasible.
Low Frequency ( $N \gg 1$ ): Executing longer action horizons (chunks) before replanning reduces compute complexity and latency. However, the robot acts on stale observations for longer periods, typically leading to degraded task success rates.

Current approaches often address this via post-hoc mechanisms (e.g., inference-time inpainting or chunk truncation) while keeping the underlying policy architecture fixed. This paper investigates whether the temporal architectural elements within the action expert itself can be optimized to improve the accuracy-complexity tradeoff, specifically by replacing standard causal self-attention with Mamba-based Selective State Space Models (SSMs).

2. Methodology

The study utilizes the SmolVLA architecture as a reference backbone, which combines a pre-trained vision-language model (VLM) with a conditional flow-matching Action Expert (AE).

Core Architecture Modification

The authors propose a direct substitution within the Action Expert:

Baseline ( $B_0$ ): Uses the original SmolVLA structure where the AE alternates between cross-attention grounding layers (connecting actions to multimodal inputs) and causal self-attention layers (modeling temporal dependencies within the action chunk).
Proposed Variant ( $M_1$ ): Retains all cross-attention grounding layers and the frozen VLM backbone but replaces the eight intra-chunk causal self-attention layers with Mamba-1 selective state-space blocks.

Technical Rationale

Inductive Bias: An action chunk is an ordered sequence where later commands must remain compatible with earlier ones. Mamba implements input-dependent state-space dynamics through causal recurrence. The authors hypothesize that this selective recurrent representation provides a more suitable inductive bias for intra-chunk coordination than the explicit causal masks used in Transformers.
Parameter Efficiency: The Mamba-based variant significantly reduces the trainable parameter count of the Action Expert from 99.9M to 76.3M (a ~24% reduction), while keeping the backbone and grounding pathways identical.
Training & Inference: Both models are trained using a conditional flow-matching objective on the LIBERO benchmark. At inference, both generate a fixed chunk of $H=50$ actions. The execution horizon $N$ is swept across $\{1, 25, 50\}$ to evaluate performance under varying replanning frequencies.

3. Key Contributions

Novel Architectural Investigation: The paper presents the first study isolating the impact of intra-chunk temporal mixing in flow-matching VLA experts by replacing Transformer self-attention with Mamba SSMs.
Horizon-Aware Evaluation: The models are rigorously evaluated across three execution horizons ( $N=1, 25, 50$ ) using the standard LIBERO benchmark suites (Spatial, Object, Goal, Long).
Demonstrated Tradeoff Improvement: The study shows that the Mamba expert significantly improves success retention as the execution horizon increases, outperforming the Transformer baseline in long-horizon scenarios.
Efficiency Gains: The Mamba variant achieves comparable performance to the Transformer baseline at $N=1$ while reducing model complexity by 24%.

4. Experimental Results

The evaluation was conducted on the LIBERO multi-task simulation benchmark (40 tasks across 4 suites) using three random seeds.

Success Rate vs. Execution Horizon

$N=1$ (Per-action replanning): The Mamba variant ( $M_1$ ) achieves a mean success rate of 76.2%, statistically matching the Transformer baseline ( $B_0$ ) at 75.9%.
$N=25$ (Moderate horizon): $M_1$ outperforms $B_0$ by 3.7% (74.7% vs. 71.0%).
$N=50$ (Full-chunk execution): $M_1$ significantly outperforms $B_0$ by 7.8% (61.8% vs. 53.9%). This represents the most critical finding, as $N=50$ corresponds to feasible real-time deployment where replanning is infrequent.

Success Retention

When moving from $N=1$ to $N=50$ :

The Transformer baseline ( $B_0$ ) suffers a performance drop of 22.0 percentage points.
The Mamba variant ( $M_1$ ) suffers a drop of only 14.5 percentage points.
This indicates that $M_1$ retains 7.5 additional points of success rate under long open-loop execution.

Per-Suite Analysis

The advantage of Mamba is not uniform:

Object Suite: Shows the largest gain, with $M_1$ trailing $B_0$ by 6.0 points at $N=1$ but surpassing it by 17.7 points at $N=50$ .
Spatial Suite: $M_1$ gains 10.0 points over $B_0$ at $N=50$ .
Goal Suite: $B_0$ maintains a slight lead (1.0 point) at $N=50$ .
Long Suite: Both models struggle, but $M_1$ maintains a stable, small lead (+4.7 points).

5. Significance and Claims

The paper claims that the superiority of the Mamba-based expert in long-horizon execution is likely due to its input-dependent causal recurrence, which offers a more effective inductive bias for coordinating ordered action positions within a chunk compared to the explicit causal attention masks of Transformers.

The significance of this work lies in demonstrating that temporal modeling within the action expert is a critical factor in VLA performance, independent of observation staleness. By adopting Mamba, the system achieves a better accuracy-complexity tradeoff:

It enables feasible real-time deployment (via longer execution horizons) without the severe accuracy degradation seen in Transformer baselines.
It reduces the computational footprint (fewer trainable parameters) while maintaining or improving performance.

The authors conclude that future work should validate these findings on physical robots and across additional VLA architectures, but the current results establish that Mamba-based SSMs are a viable and superior alternative to Transformers for intra-chunk temporal mixing in flow-matching VLA policies.