💻 computer science

A Unified Formula for Affine Transformations between Calibrated Cameras

This technical note derives a closed-form expression for the affine transformation between local image patches in two calibrated views, demonstrating that the transformation depends on the relative camera pose, image coordinates, and the local surface normal.

Original authors: Levente Hajder

Published 2026-02-09

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Levente Hajder

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are holding a camera in your left hand and another in your right, both looking at the same object in the world. This is a "stereo setup." Now, imagine you take a tiny square patch of pixels from the left image and try to figure out how that same patch looks in the right image.

Usually, if the object is flat like a wall, the patch just shifts or stretches in a predictable way. But if the object is curved, like a ball or a bumpy rock, that tiny patch gets warped in a complex way.

This paper by Levente Hajder is essentially a master recipe for predicting exactly how that tiny patch will warp.

Here is the breakdown using simple analogies:

1. The Three Ingredients

The author says that to predict this warping, you only need three specific pieces of information:

How you moved the camera: Did you turn it? Did you slide it to the side? (The paper calls this "relative pose").
Where you are looking: Which specific spot on the screen are we talking about? (The "image coordinates").
How the object is facing: Is the surface flat, tilted, or curved at that exact spot? (The "surface normal," which is like an arrow sticking straight out of the object's surface).

2. The "Magic Formula"

Before this paper, if you wanted to calculate how a patch warps, you might have needed different math formulas for different situations (e.g., one formula for flat walls, another for moving cameras, another for rotating cameras).

This paper provides a single, unified formula (Equation 5 in the text) that works for everything. It's like having one universal remote control that works on every brand of TV, instead of needing a different remote for Samsung, Sony, and LG.

The formula takes those three ingredients (movement, location, and surface angle) and mixes them together to produce a 2x2 grid of numbers (a matrix). Think of this grid as a "stretching instruction manual." It tells you exactly how to squish, stretch, or skew the tiny patch of pixels to match what the second camera sees.

3. How It Works (The Deconstruction)

The author breaks this complex math down into three simple parts added together:

The Rotation Part: This accounts for how the camera turned.
The Translation Part: This accounts for how the camera moved sideways or forward.
The Surface Part: This accounts for the shape of the object itself.

The paper shows that the final "stretching instruction" is just the sum of these three influences.

4. The "Standard Stereo" Test

To prove the formula works, the author tested it on a very simple, classic scenario: two cameras sitting side-by-side looking straight ahead (like human eyes).

In this simple case, the complex formula simplifies down to a much shorter equation.
The result matched exactly what other experts had already discovered for this specific simple case.
This proves the "Master Recipe" is correct because when you use it on a simple dish, it tastes exactly like the old, known recipe.

Summary

In short, this paper gives computer vision a single, all-purpose tool to understand how 3D shapes look different from two different angles. Instead of needing a different math trick for every type of camera movement or object shape, you can now use this one "Unified Formula" to calculate the distortion of any local image patch, provided you know how the cameras moved and how the object is oriented.

Technical Summary: A Unified Formula for Affine Transformations between Calibrated Cameras

Problem Statement
The paper addresses the fundamental computer vision challenge of characterizing the relationship between local image deformations and scene geometry. Specifically, it seeks to derive a closed-form expression for the local affine transformation that maps image patches between two calibrated views. While the relationship between planar surfaces and homographies is well-established, this work focuses on the more general case where the implicit parameters of a tangent surface (specifically its normal vector) are known, allowing for the determination of local affine parameters without assuming a global planar scene.

Methodology
The authors formulate the problem using a standard stereo vision setup involving two pin-hole cameras with known intrinsic matrices ( $K_1, K_2$ ). The relative pose between the cameras is defined by a rotation matrix $R \in SE(3)$ and a translation vector $t$ .

The derivation proceeds as follows:

Homography Foundation: The authors start with the known homography $H$ that maps points on a planar surface between two views, expressed as $H = R - \frac{tn^T}{d}$ , where $n$ is the surface normal and $d$ is the distance to the plane relative to the first image.
Differentiation: Following previous work [4], the local affine transformation matrix $A$ (size $2 \times 2$ ) is obtained by calculating the partial derivatives of the projected coordinates in the second image with respect to those in the first.
Closed-Form Derivation: By substituting the elements of the homography matrix and performing the differentiation, the authors derive explicit expressions for the elements of $A$ ( $a_{11}, a_{12}, a_{21}, a_{22}$ ). These elements are functions of the rotation matrix $R$ , translation $t$ , image coordinates ( $u, v$ ), and the surface normal $n$ .
Structural Decomposition: The final unified formula (Equation 5) is presented as the sum of three $2 \times 2$ matrices: the upper-left block of the rotation matrix, and two dyadic terms involving the translation and the surface normal.

Key Results
The primary result is a unified, closed-form equation for the affine transformation $A$ . The paper demonstrates that $A$ is strictly a function of:

The relative camera pose ( $R, t$ ).
The image coordinates of the point of interest.
The local surface normal ( $n$ ).

The authors validate this general formula by applying it to a standard stereo setup (where $R=I$ and $t=[t_x, 0, 0]^T$ ). In this specific case, the derived formula simplifies to a form consistent with results published in prior literature [5], with the distinction that the current derivation explicitly accounts for calibrated cameras.

Significance and Claims
The paper positions this unified formula as a "parent equation" capable of encompassing various special cases that have historically been treated as separate problems in the literature. The authors claim the following significance:

Generality: The formula provides a single, structured framework that encapsulates the complex interplay between camera motion, image coordinates, and surface orientation.
Geometric Interpretation: By decomposing the transformation into a sum of matrices, the work offers a clear geometric interpretation of how rotation, translation, and surface orientation individually contribute to image deformation.
Foundation for Future Work: The authors state that the primary strength of this work lies in its ability to simplify the derivation of specific models. They identify this as a consistent mathematical foundation for future geometric algorithms, ranging from robust feature tracking to direct 3D reconstruction.

Future Directions
The paper outlines a plan to systematically derive and analyze special cases using this unified formula, including:

Planar motion models (restricted to 2D planes).
Pure translation and pure rotation scenarios.
Small-baseline approximations to link with traditional optical flow and scene flow formulations.

The work concludes by asserting that this mathematical unification aims to streamline the development of future computer vision algorithms.

1. The Three Ingredients

2. The "Magic Formula"

3. How It Works (The Deconstruction)

4. The "Standard Stereo" Test

Summary

More like this