Bimanual Manipulation Within an 8 GB Budget: Zero-Copy Sensing and Quantized ACT on an Entry-Level Jetson

Imagine a world where robots aren't just clunky machines in a factory, but nimble helpers that can fold laundry, hand you a drink, or pick up a squishy beanbag with two hands at once. For a long time, teaching these robots to do such delicate tasks required "teleoperation," where a human remotely guides the robot's arms to show it how it's done. The robot then learns by copying these movements, a process called "imitation learning." But here's the catch: until now, the "brain" of these robots usually needed a massive, expensive computer—like a supercharged workstation with a giant graphics card—to think and move. This meant the robot was often tethered to a heavy, power-hungry machine, making it hard to use in real homes or on the go. The big question researchers are asking is: Can we shrink this giant brain down to fit inside a tiny, affordable computer chip that lives right on the robot itself?

This paper takes a bold step in that direction. The researchers built a two-handed robot system that runs entirely on an NVIDIA Jetson Orin Nano Super, a small, entry-level computer chip with only 8 GB of memory (about the size of a large smartphone's storage). They tested it on a tricky task: picking up a deformable beanbag with two arms and moving it to a target spot. They wanted to see if they could make the robot fast and smart enough to work without any help from a big computer.

The Three Big Surprises

The team started with three big guesses about what would be the hardest part of making this work. They expected to run into memory limits, they expected to have to choose between two different types of "brain" architectures, and they expected that making the robot faster would require sacrificing accuracy. But their experiments turned those guesses on their heads.

1. The Memory Myth: It Wasn't the RAM, It Was the CPU
The researchers thought that streaming video from three cameras (one on the robot's head, two on its wrists) would fill up the robot's 8 GB of memory, causing it to crash or drop frames. They built a special "zero-copy" pipeline to keep the video data in the graphics memory without shuffling it back and forth to the main processor, hoping this would save space.

• The Reality: They were wrong about the memory. The standard way of moving video data actually fit perfectly within the 8 GB limit and didn't drop a single frame.
• The Real Win: The "zero-copy" trick didn't save space; it saved time and brainpower. By stopping the robot's main processor from wasting energy shuffling video data, they freed up a massive amount of CPU power. The peak usage of a single processor core dropped from a sweating 98.0% down to a relaxed 77.0%. This is like taking a heavy backpack off a runner; the runner doesn't get bigger, but they can run much faster and smoother. This extra "headroom" was crucial for keeping the robot's control loop running smoothly without missing a beat.

2. The Brain Race: Speed vs. Patience
The team trained two different types of robot brains on the exact same set of demonstrations: ACT (Action Chunking with Transformers) and Diffusion Policy.

• The Setup: They gave ACT 100,000 training steps (a standard amount for this type of brain) and gave Diffusion Policy 200,000 steps (twice as much, because Diffusion usually needs more time to learn).
• The Result: ACT learned the task perfectly, succeeding in 19 out of 20 trials. Diffusion Policy, despite getting double the training time, failed to learn a usable strategy at all (0 out of 10 trials).
• The Takeaway: This wasn't because Diffusion is a "bad" brain. It's because Diffusion is expensive to train. On a tight budget, the "faster learner" (ACT) was the only one who could finish the race. The researchers emphasize that this doesn't mean Diffusion is worse forever, just that it costs too much "gradient steps" (training time) to be practical for this specific setup right now.

3. The Speed Trick: When "Good Enough" is Actually Perfect
To make the robot fast enough to react in real-time, the team converted the ACT brain to run on a specialized engine called TensorRT and tried running it with lower precision (using FP16 and INT8 math instead of the super-precise FP32).

• The Speed Boost: Lowering the precision made the robot think much faster. The time it took to make a decision dropped from 114.02 ms (milliseconds) with full precision to 17.93 ms with FP16, and down to 12.65 ms with INT8. That's nearly 9 times faster.
• The Surprise: Usually, when you make a computer "dumber" by using lower precision, it makes mistakes. But here, even with the "dumbest" math (INT8), the robot still succeeded 19 out of 20 times. The numerical errors were huge (up to 16.98% deviation in some numbers), but the robot still caught the beanbag.
• The Catch: The researchers found that whether you need this speed trick depends on how the robot plans its moves. If the robot plans a whole chunk of 100 moves at once (which it did), it can afford to be slow. But if it needs to re-plan every single step (a technique called "temporal ensembling"), then the slow full-precision version would miss its deadline, and the fast, lower-precision version becomes mandatory.

The Final Verdict

The paper concludes that you can indeed run a complex, two-handed robot on a tiny, $8 GB computer. The secret isn't just about saving memory; it's about managing the processor's workload so it doesn't get overwhelmed.

They recommend a specific setup for anyone trying to build a similar robot:

• Use the GStreamer video pipeline to free up CPU power.
• Train the ACT policy (it's faster to learn than Diffusion for this task).
• Run the policy in FP16 precision (which is fast and accurate enough) or INT8 if you need even more speed.

The most important lesson? Don't assume the biggest computer is the only way to go. Sometimes, the best way to make a robot smart is to stop it from wasting energy on things it doesn't need to do, and let it focus on the job at hand. The robot didn't need a supercomputer; it just needed a little bit of efficiency.