Technical Summary: OrthoMotion

Problem Statement

Controllable video generation requires independent control over camera motion and subject motion. However, existing methods suffer from entanglement, where controlling one axis (e.g., camera orbit) inadvertently distorts the other (e.g., subject trajectory).

The paper argues this is not merely an architectural failure but a representational ambiguity. Based on the optical flow equation for a scene point at depth $Z$, both camera translational motion and object self-motion scale with the inverse depth factor ($1/Z$). Consequently, in a 2D-conditioned setting, the split between camera-induced flow and object-induced flow is an ill-posed inverse problem (non-identifiable). Without explicit constraints, a model cannot distinguish between a moving camera and a moving object based solely on 2D image evidence, leading to inherent cross-talk.

Methodology: OrthoMotion

OrthoMotion resolves this entanglement by reframing decoupling as an operator design problem within the attention mechanism of a Diffusion Transformer (specifically Wan2.1). It introduces Geometry–Semantics Orthogonal (GSO) Attention, which routes camera and subject controls into algebraically complementary sub-operators.

1. Geometric Channel (Camera)

The camera motion is injected into the Rotary Position Embedding (RoPE) phase.

• Mechanism: Instead of additive injection or non-norm-preserving projective embeddings (which break RoPE invariants), OrthoMotion applies a norm-preserving rotation to the query and key vectors.
• Mathematical Form: A global orthogonal matrix $\Omega(g) \in SO(d_h)$ is composed with the standard RoPE rotation $R(p)$. The logit becomes $\tilde{q}_i^\top \tilde{k}_j = q_i^\top R(p_i)^\top \Phi_{ij} R(p_j) k_j$, where $\Phi_{ij}$ depends only on the relative camera pose.
• Theoretical Guarantee: By Proposition 1 and Theorem 2, this operation preserves token norms and the softmax temperature geometry, ensuring the camera control acts as a pure geometric warp without distorting the attention statistics or masquerading as a per-pixel object term.

2. Semantic Channel (Subject)

The subject motion is injected into the Cross-Attention values.

• Mechanism: Subject trajectory tokens are appended to the text keys/values and gated by a soft mask $M_t(x)$ specific to the subject region.
• Mathematical Form: The update is an additive translation in the value space: $o_i = \sum \alpha_{ij} v_j + \sum \beta_{ik} v_{\tau}^k$.
• Theoretical Guarantee: As per Proposition 3, this operation is additive and local. It does not alter the query/key geometry (the camera phase) and is strictly confined to the subject mask, preventing leakage outside the target region.

3. Decoupling Regularizer

To ensure the two channels do not interfere, the authors introduce a Decoupling Regularizer ($\mathcal{L}_\perp$).

• Objective: The regularizer minimizes the Frobenius norm of the inner product between the response Jacobians of the camera ($\hat{J}_g$) and the subject ($\hat{J}_\tau$):
  $$\mathcal{L}_\perp = \| \hat{J}_g^\top \hat{J}_\tau \|_F^2$$
• Theoretical Guarantee: Theorem 3 proves that minimizing $\mathcal{L}_\perp$ drives the response subspaces to orthogonality. This bounds the first-order cross-talk (leakage) between the two controls. Unlike prior methods that hope for disentanglement to emerge, OrthoMotion guarantees it by construction.

Key Contributions

1. Representational Diagnosis: The paper formally proves that camera-object entanglement stems from the shared $1/Z$ scaling in optical flow, making the 2D split non-identifiable without specific operator constraints.
2. GSO Attention with Regularization: It proposes a dual-channel architecture (Geometric Phase + Semantic Value Injection) and a regularizer that mathematically enforces orthogonality between their response subspaces.
3. Cross-Talk Error (CTE) Metric: A new metric is introduced to quantify leakage:
   • $CTE_{c \to s}$: Subject drift when sweeping the camera.
   • $CTE_{s \to c}$: Camera drift when sweeping the subject.
4. First Guaranteed Disentanglement: It is the first method to provide a theoretical guarantee of disentanglement rather than relying on emergent behavior.

Experimental Results

Evaluated on the Wan2.1-1.3B backbone (with generalization tested on 14B and CogVideoX-2B), OrthoMotion achieves:

• State-of-the-Art Accuracy: Simultaneously outperforms baselines (MotionCtrl, MotionPro, Tora+ReRoPE) in camera rotation/translation error and subject trajectory accuracy (ObjMC).
• Drastic Cross-Talk Reduction: Reduces $CTE_{c \to s}$ by >2.4× compared to a variant without the regularizer (11.3 px $\to$ 4.6 px).
• No Fidelity Loss: Achieves these gains without degrading FVD (Fréchet Video Distance) or CLIP similarity; in fact, it improves upon them.
• Ablation Insights:
  • Removing the regularizer ($\mathcal{L}_\perp$) significantly increases cross-talk.
  • Using non-norm-preserving injections (like additive Plücker) degrades fidelity.
  • The optimal regularization weight is found at $\lambda=0.1$.
  • Single-axis performance matches or beats specialized controllers, confirming no "joint-control penalty."

Significance and Claims

The paper claims to solve the open problem of decoupled camera-subject control by addressing the root representational ambiguity rather than just tuning architecture.

• Theoretical Shift: It moves the field from treating disentanglement as a data or training artifact to a design guarantee via operator orthogonality.
• Practical Impact: It enables cinematic creation where camera and subject can be steered independently ("two separate dials") without the "drags" and "perturbations" seen in previous methods.
• Limitations: The authors acknowledge the method currently relies on first-order Jacobian estimates and assumes rigid scenes (based on Eq. 1). Extending this to deformable subjects and multi-object scenes is identified as future work.