Does Enabling the Depth API Improve Anchor Stability? A Controlled Cross-Platform Evaluation of Mobile AR
A controlled cross-platform evaluation of 600 mobile AR runs reveals that enabling depth APIs (ARCore/ARKit) does not improve anchor stability and may even increase error, whereas scene texture contrast emerges as the dominant factor influencing spatial drift.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine trying to place a virtual object, like a floating digital cube, onto a real-world table using a smartphone. The promise of this technology is that the object stays exactly where you put it, even if you walk away and come back. This is the core challenge of markerless mobile augmented reality: keeping the digital world locked to the physical one without using physical stickers or markers to hold it in place. Currently, smartphones achieve this by watching the room with their cameras and using internal sensors to guess their own movement. However, this method has a flaw. If the room is too plain, like a white wall, or if the light is dim, the phone can lose its grip on reality, causing the virtual object to drift or slide away. To fix this, the major software platforms that power these apps have introduced a new tool: a depth sensor. This technology measures the distance to objects in the room, either by bouncing light off them or by calculating depth from the camera's movement. The intuitive hope was that adding this extra layer of distance information would act as a safety net, keeping the virtual object steady even when the camera struggles to see details.
A team of researchers set out to test if this hope was true. They did not just look at how accurate the depth sensors were in a lab; they asked a more practical question: does turning on the depth setting actually stop the virtual object from drifting? To find out, they ran six hundred controlled experiments using three different smartphones: two Android devices and one iPhone. These phones represented the full range of depth technology available to consumers today. One phone used a time-of-flight sensor that bounces light to measure distance, another relied entirely on software to calculate depth from the camera's movement, and the third used a laser scanner known as LiDAR. The researchers placed each phone on a motorized stand in the center of a room and programmed it to spin around in a wide circle, looking at the walls. Crucially, they covered the reference point on the wall with a matte sheet so the phone could not see it while it was spinning. This forced the phone to rely solely on its internal tracking to remember where it was. They tested this setup in two different room conditions: one with walls covered in bold, high-contrast patterns, and another with bare, plain walls. They also tested in bright light and in very dim light, with the depth sensor turned on in some runs and turned off in others.
The results were surprising and clear. In none of the test scenarios did turning on the depth sensor make the virtual object more stable. In fact, in almost every case, the object drifted slightly more when the depth sensor was active. This was true even for the phone with the laser scanner, which provided a complete and perfect depth map of the room on every single frame. The researchers found that the depth setting itself did not help the phone keep its place. Instead, the most important factor was the texture of the room. When the phone was in the room with the plain, bare walls, the virtual object drifted significantly, often ending up more than half a meter away from its starting point. In the room with the patterned walls, the drift was much smaller. The texture of the environment mattered far more than the depth sensor.
The study also revealed why the depth sensor failed to help in some cases. On the Android phones, the software that provides the depth map was very picky. It would only send depth data when the room had enough visual texture to work with. In the bare room, the phone simply stopped sending depth information most of the time, leaving the system with no extra data to rely on. On the iPhone, the depth data was always available, yet the object still drifted just as much as when the sensor was off. This suggests that the problem is not just about having the data, but about how the phone's internal tracking system uses it. The researchers concluded that for developers and users, focusing on the depth setting is not the solution to stability. Instead, the physical environment is the key. If a user wants their virtual content to stay put, they should ensure it is placed in a room with enough visual detail on the walls and surfaces, rather than relying on the phone's depth sensor to do the heavy lifting. The technology is ready to measure depth, but it cannot yet use that measurement to anchor a virtual object in a plain, dark room.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.