Inverting Non-Injective Functions with Twin Neural Network Regression

1. Problem Statement

The paper addresses the fundamental challenge of inverting non-injective functions. In many scientific and engineering domains (e.g., inverse kinematics, physics, chemistry), a forward mapping $f: X \to Y$ is known, but the inverse mapping $f^{-1}: Y \to X$ is ill-posed because the function is not one-to-one (non-injective).

• The Core Issue: A single output $y$ may correspond to multiple distinct inputs $x_1, x_2, \dots, x_n$.
• Failure of Traditional Methods: Standard regression algorithms (like traditional Neural Networks) trained to approximate the inverse mapping are forced to learn an average of these incompatible solutions. This results in predictions that lie "between" valid branches, yielding outputs that are not valid preimages of the target.
• Limitations of Probabilistic Methods: While probabilistic approaches (e.g., Mixture Density Networks) can model multimodal distributions, they often lack the stability, reproducibility, and computational efficiency required for deterministic pipelines (e.g., real-time robot control).

2. Methodology: Inverse Twin Neural Network Regression (ITNNR)

The author proposes Inverse Twin Neural Network Regression (ITNNR), a deterministic framework that reformulates the global inversion problem into a collection of locally invertible problems.

2.1. Core Concept: Local Anchoring

Instead of learning a global inverse function, ITNNR learns to predict local corrections relative to known "anchor" points.

• Twin Neural Network Regression (TNNR): Originally designed for regression, TNNR learns the difference between target values of paired inputs rather than the targets themselves.
• Inverse Formulation: For a target $y_{new}$, the method selects $k$ nearest neighbor anchors $(x_{anchor}, y_{anchor})$ from a dataset based on the distance in the output space ($y$).
• Prediction Mechanism: The neural network $F$ is trained to predict the difference in input space given the difference in output space and the anchor input:
  $$F(y_{new}, y_{anchor}, x_{anchor}) \approx x_{new} - x_{anchor}$$
  The final prediction is reconstructed as:
  $$\hat{x}_{new} = F(y_{new}, y_{anchor}, x_{anchor}) + x_{anchor}$$

2.2. Handling Non-Injectivity (Branch Selection)

Since multiple anchors near a specific $y_{new}$ might belong to different branches of the inverse function, ITNNR generates multiple candidate solutions (one per anchor).

• Consistency Check: To select the correct branch, the method projects the candidate $\hat{x}_{new}$ back through the forward function $f$.
• Ranking: Candidates are ranked by the consistency error $\|f(\hat{x}_{new}) - y_{new}\|$. The candidate with the lowest error is selected.
• Data Scenarios:
  • Known Forward Function: The exact mathematical formula is used for consistency checking.
  • Measured Data: A traditional neural network is first trained to approximate the forward function, which then serves as the consistency checker.

2.3. Handling Infinite Solutions

For cases where input dimensions exceed output dimensions (e.g., a 2D input mapping to a 1D output, creating a circle of solutions), ITNNR does not attempt to find all solutions. Instead, it uses the choice of anchor to bias the solution toward a specific region of the input space (e.g., the current state of a robot arm), effectively acting as a "gauge fixing" mechanism.

3. Key Contributions

1. Deterministic Framework: Introduces a fully deterministic method for resolving multi-valued inverse mappings, avoiding the stochastic nature and instability of probabilistic models.
2. Local Reformulation: Successfully reframes the global ill-posed inversion problem into a set of well-defined local regression problems anchored to known data points.
3. Branch Resolution: Demonstrates that by combining TNNR with $k$-Nearest Neighbor (k-NN) search and a consistency check, the method can accurately recover specific branches of an inverse function without averaging.
4. Versatility: The method is applicable to both synthetic mathematical functions and real-world data, handling scenarios with finite discrete solutions, continuous manifolds, and infinite solution sets.

4. Experimental Results

The author evaluated ITNNR on 9 diverse experiments, including:

• Toy Problems: 1D cubic and quartic polynomials ($f(x) = x^3 - x$).
• Geometric Problems: Inverting the upper half of a unit ball (2D $\to$ 1D).
• Robotics: Inverse kinematics for 2-link, 3-link, and 6-DOF robot arms (2D and 3D).

Performance Metrics:

• Comparison: ITNNR was compared against Traditional Neural Networks (NN), $k$-Nearest Neighbors (k-NN), and Mixture Density Networks (MDN).
• Accuracy:
  • Traditional NN: Failed to converge to meaningful solutions due to averaging.
  • k-NN: Provided a baseline but suffered from discontinuities and poor local accuracy.
  • MDN: Performed well on discrete solution sets but struggled with continuous manifolds (e.g., the half-ball problem).
  • ITNNR: Consistently achieved the lowest Root Mean Squared Error (RMSE).
• Quantitative Gains:
  • ITNNR reduced RMSE by 80–93% compared to k-NN.
  • ITNNR reduced RMSE by 46–83% compared to MDN (excluding the half-ball case where MDN struggled significantly).
  • Even with noisy data (1% Gaussian noise), ITNNR maintained a significant advantage, reducing errors by 30–85% relative to k-NN.

5. Significance and Conclusion

The paper demonstrates that ITNNR is a robust, scalable, and accurate solution for inverting non-injective functions. Its significance lies in:

• Solving the "Averaging" Problem: It fundamentally avoids the failure mode of standard regression on multi-modal targets.
• Practical Applicability: It is particularly valuable for Inverse Kinematics, where deterministic, reproducible, and stable solutions are critical for control systems, and where exact forward solvers may be too slow or unavailable.
• Data Efficiency: The method leverages the "pairwise difference" approach, effectively multiplying the training data size, making it highly effective even with smaller datasets.

The author concludes that by anchoring predictions to local regions of injectivity, ITNNR transforms a globally ill-posed problem into a collection of solvable local problems, offering a superior alternative to both traditional regression and probabilistic inversion methods.