PhaForce: Phase-Scheduled Visual-Force Policy Learning with Slow Planning and Fast Correction for Contact-Rich Manipulation

PhaForce is a phase-scheduled visuomotor policy that enhances contact-rich manipulation by coordinating a slow, vision-dominant diffusion planner with a fast, force-driven corrector to enable high-frequency, phase-aware residual corrections, achieving an 86% success rate and superior adaptability compared to existing baselines.

The Gist

Imagine you are teaching a robot to perform delicate tasks like plugging a charger into a wall socket, opening a sticky drawer, or wiping a whiteboard clean.

If you only give the robot eyes (cameras), it's like trying to thread a needle in the dark. The robot can see the hole, but it doesn't know if the plug is jammed, if it's tilted slightly, or if it's pressing too hard. It might just keep pushing until it breaks something.

If you only give the robot feel (force sensors), it's like trying to drive a car with your eyes closed, feeling the road but having no idea where you are going. It might feel the wall, but it won't know which wall or how to get around it.

PhaForce is a new "brain" for robots that combines sight and touch perfectly. It solves the problem by acting like a team of two experts working together: a Strategic Planner and a Reflexive Pilot.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Slow Brain, Fast Hands" Mismatch

Most robot brains are slow. They take a picture, think for a moment, and then decide on a big chunk of movement (like "move forward 5 inches"). This happens maybe 6 times a second.

But the robot's hands move much faster (24 times a second). If the robot hits a bump or gets stuck, a "slow brain" is too sluggish to react. It's like driving a car where you only check the road once every 10 seconds. By the time you see the pothole, you've already crashed.

Also, existing robots often use force sensors blindly. They might try to "feel" their way through a task even when they are in open space (where there is no contact), which just creates noise and confusion.

2. The Solution: The "PhaForce" Team

PhaForce introduces a Phase Schedule. Think of a task like a song with different verses and choruses. You don't play the same notes for the whole song; you change your style depending on the part of the song.

PhaForce has three main parts:

A. The "Traffic Cop" (Contact-Aware Phase Predictor)

This is the smartest part of the system. It constantly asks:

• "Are we touching anything right now?"
• "What stage of the task are we in? (e.g., Are we approaching the socket, searching for the hole, or inserting?)"

It acts like a traffic cop directing the robot. It tells the robot: "Right now, we are in the 'Search' phase, so listen to your side-to-side feelings. But once we are 'Inserting,' ignore the side feelings and focus on pushing straight down."

B. The "Strategic Planner" (The Slow Brain)

This is the main brain that plans the big moves. It looks at the camera and the force sensors, but it's very careful.

• The Trick: It uses a special filter (called Orthogonal Residual Injection). Imagine you are painting a picture (the visual plan). If you add force data, you don't want to smear the whole painting. Instead, you add a tiny, transparent layer of "force info" on top that only changes what needs to change, without ruining the original picture.
• This ensures the robot doesn't get confused by the force sensors and forgets where it's supposed to go visually.

C. The "Reflexive Pilot" (The Fast Corrector)

This is the robot's "knee-jerk reaction" system. It runs 4 times faster than the Strategic Planner.

• While the Planner is thinking about the next big move, the Pilot is constantly making tiny, micro-adjustments.
• If the robot feels a sudden bump (like the plug hitting the rim of the socket), the Pilot instantly nudges the robot to the side to find the hole, before the Strategic Planner even knows there was a problem.
• Crucially, the Traffic Cop tells the Pilot where to nudge. If we are in the "Search" phase, the Pilot only moves side-to-side. If we are in the "Insert" phase, it only moves up-and-down. This prevents the robot from making wild, confusing movements.

3. Real-World Results: Why It Matters

The researchers tested this on a real robot arm with five different tasks. Here is what happened:

• Plugging in a Charger: Old robots often got stuck at the hole entrance, jamming the plug. PhaForce realized it was stuck, switched to "Search Mode," and wiggled the plug until it found the hole.
• Opening a Sticky Drawer: The robot felt the friction and adjusted its pull smoothly, rather than yanking the handle and breaking it.
• Wiping a Board (The "Surprise" Test): They moved the board 3 inches higher than the robot had ever seen before.
  • Old Robots: Crashed into the board, pressed too hard, or couldn't wipe at all.
  • PhaForce: Felt the board was higher, instantly adjusted its pressure, and wiped the board perfectly clean.

The Big Picture

Think of PhaForce as a master chef who is also a sous-chef.

• The Chef (Slow Planner) decides the recipe and the big steps: "Chop the onions, then sauté them."
• The Sous-Chef (Fast Corrector) is right there at the stove, tasting the sauce every second. If it's too salty, the Sous-Chef adds water immediately.
• The Recipe Card (Phase Schedule) tells them exactly when to taste and what to adjust.

By combining a slow, smart plan with fast, smart reflexes, and knowing exactly when to use which sense, PhaForce allows robots to handle messy, real-world tasks with a level of dexterity that was previously impossible. It's not just about seeing; it's about feeling the right thing at the right time.

Technical Summary

Here is a detailed technical summary of the paper "PhaForce: Phase-Scheduled Visual–Force Policy Learning with Slow Planning and Fast Correction for Contact-Rich Manipulation."

1. Problem Statement

Contact-rich manipulation tasks (e.g., insertion, wiping, drawer opening) require more than just visual alignment; they demand precise closed-loop reactions to force/torque (F/T) transients. Existing generative visuomotor policies (like Diffusion Policies) face two critical limitations:

• Timescale Mismatch (Gap-1): Generative policies typically operate at low frequencies due to inference latency and action chunking. However, F/T signals are high-frequency feedback essential for reacting to short-horizon events (e.g., stick-slip, jamming, micro-impacts). Relying solely on low-rate updates leads to under-reaction to these critical transients.
• Lack of Explicit Phase Scheduling (Gap-2): Existing force-aware methods often fuse force and vision indiscriminately. They lack a mechanism to determine when, how much, and where (in which corrective subspace) to apply force. Contact-rich tasks are inherently multi-phase (e.g., searching vs. inserting), and applying high-frequency corrections in the wrong subspace (e.g., correcting lateral motion during vertical insertion) can degrade performance or cause jamming.

2. Methodology: PhaForce Architecture

PhaForce proposes a phase-scheduled slow–fast policy that unifies low-frequency generative planning with high-rate residual correction, coordinated by an explicit contact/phase schedule. The system consists of three core components:

A. Contact-Aware Phase Predictor (CAP)

• Function: A lightweight network that estimates a continuous contact probability ($p_c$) and a soft phase belief distribution ($p$) over $K$ task-defined phases (e.g., approach, search, insert, recovery).
• Input: Multi-view RGB images, wrench history, and proprioception.
• Training: Trained for anticipation rather than instantaneous judgment. It predicts whether contact will occur in the future window and the future task phase, using automatically generated labels from wrench signals and TCP poses.
• Role: Provides the semantic schedule signal that gates force usage in both the Slow and Fast modules.

B. Slow Diffusion Planner (Low-Rate, $f_s$)

• Function: Generates a nominal action chunk (e.g., 16 steps) at a low frequency (e.g., 6 Hz).
• Dual-Gated Fusion:
  • Contact Gate ($p_c$): Controls the overall injection strength of force data. If no contact is expected, force influence is suppressed to prevent noise interference.
  • Phase Gate: Modulates the attention heads in the cross-attention mechanism based on the current phase belief ($p$).
• Orthogonal Residual Injection (ORI): Instead of overwriting visual features with force data, the force information is injected as a residual component orthogonal to the visual token. This preserves the vision-dominant task semantics while conditioning the plan on force, preventing semantic drift.

C. Fast Residual Corrector (High-Rate, $f_c$)

• Function: Runs at the control frequency (e.g., 24 Hz) to perform within-chunk micro-adjustments. It predicts a small delta pose ($\Delta T$) based on the Slow planner's base action and current F/T feedback.
• Phase-Routed Corrective Subspaces:
  • The corrector does not apply corrections blindly. It uses the phase belief ($p$) to softly route corrections into phase-specific subspaces (defined by binary masks).
  • Example: In a "search" phase, corrections are routed to planar ($x, y$) and yaw channels; in an "insert" phase, corrections are routed to normal ($z$) and yaw channels.
• Physical-Prior Supervision: The corrector is trained to minimize the error against a "virtual target pose" derived from physical priors (e.g., relieving tangential friction during search, maintaining normal force during wiping). This allows the model to learn corrective trends without explicit human annotation of the target pose.

3. Key Contributions

1. Unified Framework: Introduces PhaForce, the first framework to explicitly coordinate long-horizon generative planning and high-rate residual correction using a unified contact/phase schedule.
2. Explicit Scheduling Signal: Proposes a mechanism (Contact Probability + Phase Belief) that dynamically decides:
   • When to trust force (gating by contact probability).
   • How much to fuse force (modulating attention).
   • Where to correct (routing to specific corrective subspaces).
3. Orthogonal Residual Injection: A novel fusion technique in the Slow planner that integrates force without degrading the semantic integrity of visual features.
4. Phase-Routed Correction: A method for the Fast corrector to apply high-frequency adjustments only in relevant degrees of freedom, preventing spurious corrections that lead to jamming.

4. Experimental Results

The method was evaluated on a Flexiv Rizon 4s robot across five real-world contact-rich tasks: Charger Plug-in, USB Plug-in, Drawer Opening, and Wiping (both In-Distribution and Out-of-Distribution).

• Success Rate (SR): PhaForce achieved an average SR of 86%, a +40 percentage point improvement over strong baselines (Diffusion Policy, Force-Concat, and Reactive Diffusion Policy).
• Task-Specific Performance:
  • Plug-in Tasks: Significantly reduced failures caused by stagnation (getting stuck at the rim) and partial insertion. The phase belief enabled timely "retreat-and-retry" behaviors.
  • Wiping (OOD): When the target board height was shifted by 3cm (OOD), standard chunk-based planners failed completely (0% SR) due to over-pressing or detachment. PhaForce maintained 85% SR by using the Fast corrector to compensate for the geometric shift via force feedback.
• Contact Quality: In wiping tasks, PhaForce achieved the highest wiping score and the lowest rates of over-pressure and under-pressure compared to baselines, demonstrating superior force regulation.
• Ablation Studies:
  • Removing Phase Belief caused a massive drop in SR (85% $\to$ 25% for USB), proving the necessity of explicit phase scheduling.
  • Removing Orthogonal Residual Injection degraded performance, confirming the need to preserve visual semantics.
  • Removing the Fast Corrector caused total failure in OOD scenarios, highlighting the critical role of high-rate residual correction.

5. Significance

PhaForce addresses a fundamental bottleneck in robotic manipulation: the gap between the slow, semantic nature of generative vision models and the fast, reactive nature of physical contact. By introducing an interpretable, phase-aware scheduling mechanism, the paper demonstrates that force feedback can be effectively utilized without compromising visual planning. This approach enables robots to handle complex, multi-stage contact tasks with high robustness, particularly in out-of-distribution scenarios where visual cues alone are insufficient. It sets a new standard for integrating physical feedback into modern imitation learning pipelines.