Towards Symmetry-sensitive Pose Estimation: A Rotation Representation for Symmetric Object Classes

1. Problem Statement

The paper addresses a fundamental challenge in 6D object pose estimation: rotational symmetry.

• The Ambiguity Issue: Many industrial and daily objects (e.g., cylinders, cubes, cans) possess rotational symmetries. For these objects, multiple distinct 3D orientations ($p_1, p_2, \dots$) produce visually identical inputs ($V(p_1) = V(p_2)$).
• The Learning Failure: Standard deep learning approaches assume a bijective relationship between visual input and numeric output. When training a network to predict rotation (e.g., via Euler angles, quaternions, or rotation matrices) for symmetric objects, the network faces a multi-modal distribution problem. It cannot learn a unique mapping because different numeric labels correspond to the same visual input.
• Limitations of Current Solutions:
  • Discretization: Splitting the rotation space into bins requires multiple networks and classifiers, increasing complexity and training time.
  • Symmetry-Invariant Metrics: Using metrics that accept any of the $n$ correct poses (e.g., VSD, MSSD) hides the ambiguity problem, preventing networks from learning a specific, continuous canonical pose.
  • Clamping: Restricting angles to a canonical range (e.g., $[0, 180^\circ]$) creates discontinuities at the boundaries (e.g., $179^\circ$ and $1^\circ$ are visually similar but numerically far apart), which hinders gradient-based optimization.

2. Methodology: SARR (Symmetry-Aware Rotation Representation)

The authors propose SARR, a novel rotation representation that modifies trigonometric identities to be unique and continuous with respect to the object's specific symmetry properties.

A. Formal Definition

Instead of using raw angles or standard representations, SARR maps the rotation space to a continuous, unique space based on the object's symmetry degree ($\kappa$) per axis.

• Symmetry Vectors: For an object, the symmetry is defined by a vector $\kappa = [\kappa_\alpha, \kappa_\beta, \kappa_\gamma]$, representing the number of visually identical poses within a $360^\circ$ rotation for each axis.
  • Example: A cylinder with 2-fold symmetry around the Z-axis has $\kappa_z = 2$.
• The Representation: SARR replaces standard trigonometric functions with frequency-scaled versions. For an angle $\theta$ and symmetry degree $\kappa$, the representation uses:
  $$\sin(\kappa \cdot \theta) \quad \text{and} \quad \cos(\kappa \cdot \theta)$$
  This effectively "wraps" the rotation space $\kappa$ times, ensuring that visually identical poses map to the exact same numeric vector, while poses that are visually similar (near symmetry boundaries) map to numerically close vectors.

B. Handling Continuous Symmetry

For objects with continuous symmetry (e.g., a perfect cylinder where rotation around the Z-axis is infinite), the paper sets the frequency parameter $\lambda = 0$, effectively removing the dependency on that specific angle in the representation, as it is irrelevant to the visual appearance.

C. Inverse Mapping

To evaluate results against ground truth, an inverse mapping is provided to convert the SARR vector back to canonical Euler angles ($\alpha|_C, \beta|_C, \gamma|_C$). This ensures the output is a unique, canonical pose suitable for evaluation.

3. Key Contributions

1. Formal Problem Description: The authors rigorously define the ambiguity problem in symmetric pose estimation, distinguishing between visual equality and numeric uniqueness, and highlighting the failure of standard continuity assumptions.
2. SARR Representation: Introduction of a generic, continuous, and unique rotation representation that adapts to specific object symmetries (discrete and continuous) without requiring space discretization or multiple networks.
3. Symmetry-Sensitive Evaluation: The use of ARC (Average Recall Cosine), a metric that penalizes ambiguity by requiring the prediction to match a specific canonical pose, rather than accepting any valid symmetric pose.
4. Comprehensive Evaluation: Extensive experiments on the T-LESS and ITODD datasets (part of the BOP benchmark), demonstrating that SARR outperforms standard representations (Euler, Quaternions, 6D, etc.) even when those representations are trained on canonical ground truth.

4. Experimental Results

The authors trained modified CenterNet architectures on T-LESS and ITODD using depth images (and grayscale RGB for ablation).

• Performance vs. Standard Representations:

  • Networks trained with SARR significantly outperformed those trained on Euler angles, Rotation Matrices, Quaternions, 6D representations, and standard trigonometrics.
  • Under the strict ARC metric (symmetry-sensitive), SARR achieved the highest scores. For example, on T-LESS, SARR achieved an ARC of 45.0% (object scope) compared to ~10-30% for other representations.
  • Even under symmetry-invariant metrics (ARB, ARG), SARR performed competitively or better, proving that resolving ambiguity helps general performance.

• Comparison with State-of-the-Art:

  • On T-LESS, SARR outperformed existing methods (including those using 3D models and complex loss functions) under the ARC metric.
  • On ITODD, SARR outperformed the SC6D method (which also handles symmetry) under the strict ARC metric.

• Ablation Studies:

  • Scope: Training a single network per symmetry class (5 classes for T-LESS) performed better than training one per object or one for the whole dataset.
  • Input Modality: The method works well with depth data. Performance dropped significantly when using grayscale RGB, likely due to domain shift between synthetic training and real grayscale testing.
  • Symmetry Classification: Adding a task to predict the symmetry class (Dataset* scope) slightly improved or maintained orientation estimation performance, suggesting the classification task aids the optimizer.

5. Significance and Impact

• Solving the "Mean" Problem: By providing a continuous, unique target, SARR prevents neural networks from converging to the "mean" of ambiguous poses, a common failure mode in symmetric object estimation.
• No 3D Models Required: Unlike many state-of-the-art methods that rely on 3D CAD models for rendering or correspondence matching, SARR operates purely on the numeric representation of the rotation, requiring only depth or texture-less images.
• Generalizability: The approach is generic and can be extended to any object symmetry class by simply defining the symmetry vector $\kappa$.
• Evaluation Shift: The paper advocates for the use of symmetry-sensitive metrics (like ARC) to drive research toward truly resolving ambiguities, rather than relying on metrics that allow "lazy" solutions where any symmetric pose is accepted.

In conclusion, the paper demonstrates that modifying the numeric representation of rotation to align with physical symmetries is a more effective and elegant solution than modifying network architectures or loss functions, leading to state-of-the-art performance in symmetric 6D pose estimation.