💻 computer science

PRISM: Polynomial Representations for Interaction-Structured Motor Control

The paper introduces PRISM, a polynomial-based policy representation that explicitly models interaction structures among physical variables to achieve superior performance in robot locomotion and manipulation tasks compared to standard MLPs, while enabling sensorless compliant behavior without requiring explicit force or contact inputs.

Original authors: Seung Hyun Lee, Stella X. Yu

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Seung Hyun Lee, Stella X. Yu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Secret Language of Moving Machines

Imagine you are trying to teach a robot to walk or pick up a delicate egg. In the world of robotics, this is usually done by giving the robot a "brain" made of a standard type of computer program called a neural network. Think of this brain like a very smart, but slightly literal, student. It looks at what the robot's sensors see—like "my left leg is at this angle" or "my arm is moving at this speed"—and then decides what to do next.

For a long time, scientists assumed that if you just made this student's brain bigger and gave it more memory (more "parameters"), it would eventually figure out how to move perfectly. The idea was that a big enough brain could learn any trick, no matter how complicated. However, there is a catch. The real world isn't just a list of separate facts; it's a web of relationships. For example, the power of a moving leg isn't just about how fast it goes; it's about how fast it goes multiplied by how hard the muscles are pushing. These "multiplicative" relationships are the hidden physics that make movement work. If a robot's brain only looks at the facts one by one, it has to work incredibly hard to guess how they fit together. This paper asks a simple question: What if we just taught the robot's brain to understand these relationships directly, instead of making the brain bigger?

The Paper's Big Idea: PRISM

The researchers behind this study, from the University of Michigan, introduced a new way to build robot brains called PRISM (Polynomial Representations for Interaction-Structured Motor Control). Instead of forcing a standard, massive brain to guess how physical variables interact, PRISM gives the robot a special "interaction module."

Think of a standard robot brain like a chef who is given a list of ingredients (position, speed, command) and has to guess the recipe for a perfect dish. If the dish requires mixing ingredients in a specific way (like baking soda reacting with vinegar), the chef has to figure out that reaction from scratch by tasting millions of failed cakes. PRISM is like giving that chef a pre-mixed "reaction kit." It doesn't just list the ingredients; it explicitly shows how they combine. It takes the robot's observations and immediately calculates the "products" of those observations—like multiplying speed by force—so the robot can see the hidden physics right away.

The team tested this idea in two very different worlds:

Walking Robots: They used a simulated humanoid robot in a gym called Humanoid-Gym. The robot had to walk forward and stay balanced using only its own internal sensors (proprioception), without any cameras or force sensors.
Robot Arms: They tested it on a robot arm in the LIBERO environment, where the robot had to perform tasks like turning a stove knob or stacking blocks. These tasks require "compliance," meaning the robot needs to be soft and gentle when it touches things, rather than hitting them hard.

What They Found

The results were surprisingly clear. In the walking tests, the PRISM robot didn't just walk better; it walked smarter. It stayed upright for much longer (improving its survival rate to 92.50% compared to 51.00% for the standard robot) and followed walking commands with much less wobbling.

Here is the most important part: The researchers compared PRISM to a standard robot brain that was made larger to have the exact same amount of computing power. Even though the "big brain" robot had just as many parameters, it still failed to catch up. The standard robot fell over or drifted off course, while PRISM stayed steady. This suggests that simply making a robot's brain bigger isn't the answer; giving it the right structure to understand how its movements interact is what matters.

In the robot arm tests, PRISM showed a magical ability called "sensorless compliance." Usually, for a robot to be gentle, it needs special sensors to feel how hard it is pushing. PRISM didn't have these sensors. Yet, when it touched a heavy object, it naturally slowed down and yielded, just like a human hand would. It avoided the "hard impacts" that caused other robots to crash or drop things. It achieved a 91.0% success rate in tasks, beating both the standard robot and a robot that used a specialized, complex controller designed just for being gentle.

Why This Matters

The authors suggest that PRISM works because it stops the robot from having to "guess" the physics of the world. By making the interactions between variables (like speed and torque) explicit and learnable, the robot can discover things like "slip" or "impact" on its own, even without force sensors.

However, the paper is careful to note that this isn't a magic wand for every problem. The system works best when the robot has good data about its own body and movements. If the robot is missing crucial information—like not being able to see an obstacle or not knowing the weight of an object—PRISM can't fix that. It also relies on simulations for these tests, so while the results are promising, they are currently proven in the digital world.

Ultimately, the paper argues that polynomial representations (math that explicitly handles these interactions) should become a standard tool for building robots. It's a shift from "let's make the brain bigger" to "let's make the brain understand how the world works." As the authors put it, the structure of the interaction is just as important as the size of the network.

Technical Summary: PRISM – Polynomial Representations for Interaction-Structured Motor Control

1. Problem Statement

Standard robot policies, typically implemented as Multi-Layer Perceptrons (MLPs), map observations (joint positions, velocities, commands, etc.) directly to actions. While MLPs can theoretically approximate complex functions, they treat inputs as first-order variables. However, many critical physical quantities governing robot behavior—such as joint power, inertial effects, slip, contact impulse, and compliance—arise from multiplicative interactions (products) among these observable variables.

In standard MLPs, these interaction cues remain latent. The network must implicitly discover these relationships from data, often requiring excessive capacity to approximate what is physically a second-order or higher-order relationship. Simply increasing the number of parameters (model capacity) does not change the input basis; it leaves the policy without an explicit "interaction basis" aligned with the underlying physics. Furthermore, achieving compliant behavior in contact-rich tasks usually requires explicit force sensors, wrench inputs, or specialized admittance controllers, which are often unavailable in sensor-minimal deployment scenarios.

2. Methodology: PRISM

The authors introduce PRISM (Polynomial Representations for Interaction-Structured Motor Control), a policy representation that makes polynomial interactions among observable physical variables explicit, learnable, and compact.

Core Architecture

PRISM replaces the standard linear conditioning pathway for proprioceptive inputs with a factorized polynomial module. Instead of explicitly enumerating all monomial terms (which is computationally inefficient), PRISM uses learned projections to form compact latent factors.

Given a proprioceptive input vector $x_t$ , PRISM computes two learned affine factors:
$u_t = W_1 x_t + b_1$
$v_t = W_2 x_t + b_2$

The second-order representation is constructed as:
$\psi_2(x_t) = u_t \odot (1 + \alpha_2 \odot v_t) = u_t + \alpha_2 \odot (u_t \odot v_t)$
where $\odot$ denotes element-wise multiplication and $\alpha_2$ is a learnable parameter initialized near zero. This design allows the policy to retain linear features where sufficient while learning to strengthen quadratic interactions where beneficial.

The framework extends recursively to degree $K$ :
$\psi_k(x_t) = \psi_{k-1}(x_t) \odot [1 + \alpha_k \odot (W_k x_t + b_k)]$
This preserves all lower-order pathways while adding higher-order interaction terms without explicit expansion.

Integration into Learning Paradigms

Reinforcement Learning (RL): PRISM augments the actor network by applying the polynomial module to deployment-available proprioceptive states before they enter the standard MLP backbone. The action space, reward function, and low-level controllers remain unchanged.
Imitation Learning (IL): In Diffusion Policies, PRISM replaces the standard linear proprioceptive conditioning layer with the polynomial interaction layer. This layer is trained end-to-end with the policy, combining with visual and language features for action prediction.

Crucially, PRISM requires no additional sensors (force, tactile, wrench) and no privileged physical inputs (contact labels, terrain friction) during deployment.

3. Key Contributions

Identification of a Limitation: The paper identifies that default MLP policies receive only first-order physical variables, failing to explicitly represent the multiplicative interactions essential for control (e.g., torque $\times$ velocity for power).
PRISM Architecture: A compact, factorized polynomial policy representation that makes these interactions explicit and learnable without enumerating all monomials. It supports gradual activation in RL and can replace linear conditioning in Diffusion Policies.
Empirical Validation: Demonstrating that structured interaction modeling yields performance gains over both standard MLPs and larger MLPs with matched capacity, proving that interaction structure cannot be replaced by parameter count alone.

4. Experimental Results

The authors evaluated PRISM on Humanoid-Gym (locomotion) and LIBERO (contact-rich manipulation).

Humanoid Locomotion (Humanoid-Gym)

Performance: PRISM significantly outperformed both a standard MLP and a "Larger MLP" (with nearly double the parameters of the baseline) in episode length, linear/yaw velocity tracking error, and survival rate.
Stability: PRISM showed tighter convergence bounds across training seeds and reduced velocity drift compared to baselines.
Conclusion: The performance leap is driven by the explicit modeling of multiplicative interactions, not merely increased model capacity.

Contact-Rich Manipulation (LIBERO)

Compliance without Sensors: PRISM achieved a 91.0% success rate, significantly outperforming the standard Diffusion Policy (63.8%) and a sensorless compliance controller (MCC-Sensorless, 47.8%).
Behavior: PRISM exhibited emergent compliant behavior: it maintained high-speed approach in free space but instantly suppressed velocity upon contact, regulating forces without force/tactile inputs. This outperformed MCC-Sensorless, which suffered from delayed responses and high impact forces.
Backbone Agnosticism: When integrated into stronger backbones (BFM-Zero for locomotion, SmolVLA for manipulation), PRISM continued to outperform larger-capacity controls, confirming the benefit is architectural, not just capacity-based.

Physical Quantity Analysis

Linear Probing: Linear probes trained on frozen PRISM representations predicted mechanics-inspired quantities (slip velocity, joint power, contact impulse, contact work) with significantly lower Mean Squared Error (MSE) and higher Pearson Correlation Coefficients (PCC) than baselines.
Emergent Interactions: Ablation studies revealed that the most impactful learned factors corresponded to physically meaningful couplings, such as "velocity-memory" (current velocity $\times$ past velocity) and "joint-velocity coupling," confirming the model learns task-relevant physical structures.

5. Significance and Claims

The paper argues that polynomial representations should become a standard architectural choice for embodied motor control.

Physics-Aligned Inductive Bias: By making interaction structures explicit, PRISM aligns the policy's inductive bias with the physical reality of robot dynamics, where control often depends on products of state variables.
Sensorless Compliance: PRISM demonstrates that compliant, contact-aware behavior can be learned purely from proprioception and visual inputs, eliminating the need for expensive or fragile force/tactile sensors in deployment.
Efficiency over Capacity: The results suggest that improving policy performance is not solely a function of scaling model size; rather, explicitly modeling the interaction structure of physical variables yields superior data efficiency and robustness.

The authors conclude that while PRISM assumes key cues are captured by low-degree polynomials and may struggle with long-horizon history or unobserved geometry, it offers a robust, deployable solution for interaction-structured control across diverse robot morphologies and tasks.