💻 computer science

Long-Range depth estimation using learning based Hybrid Distortion Model for CCTV cameras

This paper proposes a hybrid distortion model that combines extended conventional camera models with neural network-based residual corrections to overcome the limitations of existing methods, enabling accurate 3D object localization for CCTV cameras at distances up to 5 kilometers.

Original authors: Ami Pandat, Punna Rajasekhar, G. Aravamuthan, Gopika Vinod, Rohit Shukla

Published 2026-08-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ami Pandat, Punna Rajasekhar, G. Aravamuthan, Gopika Vinod, Rohit Shukla

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to measure the distance to a ship on the horizon or a drone flying high above a city using nothing but two standard security cameras. For decades, this has been a frustrating puzzle for engineers. While cameras are excellent at seeing what is right in front of them, they struggle to judge depth when objects are far away. This is because the lenses in these cameras are not perfect; they bend light in subtle, complex ways that create distortions, especially near the edges of the image. To turn a flat picture into a three-dimensional map, computers need to know exactly how the lens bends that light. Traditional methods work well for objects a few hundred meters away, but as the distance grows to several kilometers, the tiny errors in these calculations multiply, causing the computer to guess wildly wrong about where an object actually is.

A team of researchers in India has developed a new way to solve this problem, allowing standard surveillance cameras to accurately locate objects up to five kilometers away. Their work bridges the gap between simple photography and high-precision mapping, offering a practical solution for border security, maritime monitoring, and autonomous driving without the need for expensive, power-hungry equipment like radar or laser scanners.

The core of the challenge lies in how cameras see the world. A perfect camera would act like a simple pinhole, projecting a straight line from the object to the image. Real lenses, however, are curved and imperfect, causing straight lines in the real world to appear curved in the photo. This is known as lens distortion. For short distances, standard mathematical formulas can correct this curve with enough accuracy. But for long-range viewing, these formulas are too simple. They miss the subtle, higher-order bends in the light path, leading to significant errors. The researchers found that simply trying to replace these old formulas with a powerful computer learning system, known as a neural network, did not work. When they let the computer learn the distortion from scratch, the system failed to settle on a correct answer, essentially getting lost in the complexity of the math.

To fix this, the team created a hybrid approach that combines the reliability of old methods with the flexibility of modern learning. First, they took the standard mathematical model and expanded it, adding many more variables to account for the complex ways light bends in their specific cameras. This extended model was much better than the old one, but it still left some errors, particularly when looking at objects near the edge of the camera's view or at extreme distances. The researchers then added a second layer: a small, specialized learning system designed not to replace the math, but to correct the small mistakes the math still made. Think of this as a master craftsman who has a very good rulebook for building a table, but who also has a trained eye to spot and fix the tiny imperfections that the rulebook misses.

In their experiments, the team set up two identical surveillance cameras ten meters apart, mimicking the way human eyes are spaced to see depth. They tested their system by trying to locate specific points in the landscape at distances ranging from a few hundred meters to five kilometers. Using only the standard mathematical model, the system could accurately place objects up to about 250 meters away. Beyond that, the estimated locations began to drift significantly, sometimes placing a target hundreds of meters off course. When they used their new hybrid model, the results improved dramatically. The system successfully located objects at distances of up to five kilometers with a level of precision that made the difference between a vague guess and a reliable position.

The researchers also discovered that the quality of the camera lens itself played a major role. They compared standard security camera lenses, which are designed for general use, against high-end machine vision lenses. The expensive lenses produced much cleaner images with less distortion, but even with those, the standard mathematical models failed at long ranges. This proved that the problem was not just the hardware, but the way the software interpreted the images. By refining the software to better understand the specific quirks of the lenses, they could achieve high accuracy even with the more affordable, common security cameras found in cities and ports.

The final step in their process involved translating the camera's view into real-world geography. Once the system calculated the three-dimensional position of an object, it converted those coordinates into latitude and longitude, the same system used by GPS. This allowed the researchers to plot the exact location of a distant object on a digital map. The results showed that the hybrid model kept the plotted points tightly clustered around their true locations, whereas the older methods scattered them widely. This capability means that security systems could potentially track a drone or a boat at a distance of five kilometers and know exactly where it is, without needing to move the camera or use expensive sensors.

This research demonstrates that we do not always need to build new, expensive hardware to solve difficult engineering problems. Sometimes, the solution lies in teaching our computers to understand the limitations of the tools we already have. By combining established mathematical rules with a smart, learning-based correction, the team has extended the useful range of everyday surveillance cameras by a factor of twenty. This opens the door for more affordable and accessible long-range monitoring systems, capable of keeping watch over vast distances with a clarity that was previously out of reach.

Technical Summary: Long-Range Depth Estimation Using a Learning-Based Hybrid Distortion Model for CCTV Cameras

Problem Statement
Accurate 3D localization and depth estimation for long-range photogrammetry (distances exceeding hundreds of meters) remain challenging when using standard stereo-vision systems, particularly with cost-effective CCTV surveillance cameras. While active sensing technologies like LiDAR and RADAR exist, they suffer from range limitations, high costs, power consumption, or low spatial resolution. Optical stereo-vision offers a passive, cost-effective alternative but is currently limited to ranges of approximately 100–300 meters. This limitation stems primarily from the inadequacy of conventional lens distortion models (e.g., Brown–Conrady, Kannala-Brandt) to capture complex, higher-order non-linearities in medium-quality lenses over long distances. Furthermore, direct application of neural networks to model distortion functions has been observed to fail in converging to stable camera parameter estimates. Small errors in disparity (e.g., 0.1 pixels) can result in massive depth estimation errors at long ranges (e.g., ~156 meters error at 5 km), necessitating sub-pixel accuracy and robust distortion modeling.

Methodology
The paper proposes a framework for long-range 3D localization using a stereo-camera setup with a 10-meter baseline, employing two identical surveillance cameras. The methodology integrates classical photogrammetry with learning-based refinement:

Standard Calibration and Baseline Analysis:
- The system utilizes Zhang's planar calibration method to estimate intrinsic and extrinsic parameters using a checkerboard pattern.
- A standard 14-parameter distortion model (including radial, tangential, rational, thin prism, and tilted sensor parameters) is initially applied.
- Experiments reveal that while this model achieves low reprojection errors (~0.2 pixels) during calibration, 3D localization accuracy degrades significantly beyond 250 meters, with estimated GIS coordinates diverging from true values.
Extended Distortion Model:
- To address the limitations of the 14-parameter model, the authors extend the distortion vector to include higher-order terms, systematically increasing the parameter count from 14 to 24.
- Through empirical evaluation of Root Mean Square (RMS) errors on test points, a 20-parameter distortion model was identified as optimal, balancing model complexity and overfitting. This reduced the maximum reprojection error for long-range test points from 10.62 pixels (14-parameter) to 7.21 pixels.
Hybrid Learning-Based Distortion Model:
- Recognizing that even the extended 20-parameter model leaves residual non-linearities, the authors introduce a hybrid approach.
- A neural network (NN) is trained to predict additive residual corrections ( $\delta A$ and $\delta K$ ) for the intrinsic matrix and distortion parameters derived from the 20-parameter model.
- The final parameters are calculated as $A = A^* + \delta A$ and $K = K^* + \delta K$ , where $A^*$ and $K^*$ are the outputs of the extended classical model.
- An inverse neural network is simultaneously trained to undistort image points prior to triangulation.
- The network is optimized using a Mean Squared Error (MSE) loss function that minimizes the difference between original image points and reprojected points.
3D Localization Pipeline:
- Sub-pixel accurate point correspondences are obtained using a heuristic up-sampling approach (cropping, 10x up-scaling, manual marking, and down-scaling).
- Triangulation is performed using the corrected projection matrices.
- Resulting 3D coordinates are transformed into GIS (latitude/longitude) coordinates for visualization.

Key Contributions

Hybrid Distortion Framework: The paper presents a novel hybrid model that combines a classical, extended 20-parameter distortion model with a neural network-based residual correction. This approach overcomes the convergence issues of standalone neural networks for distortion modeling while capturing non-linearities missed by classical models.
Long-Range Capability: The framework demonstrates the capability to estimate 3D positions of objects at distances up to 5 kilometers using standard CCTV surveillance cameras, a significant improvement over the typical 100–300 meter limit of existing stereo systems.
Empirical Validation of Lens Quality: The study highlights the impact of lens quality on reprojection error, noting that surveillance lenses exhibit higher errors (~~0.2 pixels) compared to machine vision lenses (~~0.07 pixels), yet the proposed method effectively compensates for these imperfections.
Sub-Pixel Precision Strategy: The implementation of a specific up-sampling heuristic to achieve sub-pixel accuracy in point correspondence, which is critical for minimizing depth errors at long ranges.

Results

Reprojection Error Reduction: The hybrid model achieved a maximum reprojection error of 6.93 pixels across 7 test points at long ranges, outperforming both the standard 14-parameter model (10.62 pixels) and the extended 20-parameter model (7.21 pixels).
GIS Localization Accuracy: Visual and numerical analysis of test points (ranging from 100m to 5km) showed that the hybrid model significantly reduced the divergence of estimated GIS coordinates compared to the true values. While the 20-parameter model showed divergence beyond 1km, the hybrid model maintained accuracy up to 5km, with points 6 and 7 (near 5km) converging closer to their original locations.
Spatial Sensitivity: The results confirmed that edge pixels suffer more from distortion than center pixels; the hybrid model effectively mitigated these edge effects, which are critical for wide-field-of-view surveillance.

Significance and Claims
The paper claims that the proposed hybrid distortion model offers a practical solution for calibrating CCTV cameras for long-range photogrammetry applications. By integrating learning-based residual correction with conventional models, the system enables reliable 3D localization using low-cost, medium-quality surveillance hardware. The authors assert that this approach bridges the gap between the high cost of active sensors (LiDAR/RADAR) and the range limitations of traditional optical stereo systems, making it suitable for applications such as border surveillance, maritime monitoring, and drone detection where long-range passive sensing is required. The study concludes that this framework significantly improves localization accuracy without requiring expensive hardware, provided that the hybrid calibration pipeline is employed.

Technical Summary: Long-Range Depth Estimation Using a Learning-Based Hybrid Distortion Model for CCTV Cameras

More like this