Investigating the performance of novel class discovery (NCD) approach for semantic segmentation on realworld SLAM LiDAR point cloud data

In the modern world of mapping, a technology called LiDAR has become a powerful tool for seeing the three-dimensional shape of our surroundings. By firing rapid pulses of light and measuring how long they take to bounce back, these sensors create vast clouds of millions of individual points, each representing a specific spot on a building, a tree, or the ground. While this data is incredibly detailed, making sense of it automatically is a difficult task. Computers are often trained to recognize only the objects they have been shown before, such as roads, houses, or trees. If the computer encounters something new, like a unique type of fence or a pile of construction debris, it often forces that object into one of its known categories, even if the fit is poor. This limitation means that in complex, real-world environments, automated maps can miss important details or misidentify them entirely. To solve this, researchers are exploring a method called novel class discovery, which aims to teach computers not just to recognize what they know, but to notice when something new appears and group those new things together on their own.

A team of researchers at Universitas Gadjah Mada in Indonesia set out to test this idea using data captured from a handheld device that moves through an outdoor environment. Unlike stationary scanners, this device, known as a SLAM LiDAR system, allows a person to walk through a campus while the sensor simultaneously maps the area and tracks its own position. This creates a continuous, flowing record of the world, but it also introduces challenges like shifting point densities and slight errors in position that do not occur in controlled laboratory settings. The researchers focused on a specific area of the university campus, covering about eighteen hectares, which contained a mix of buildings, vegetation, and paved surfaces. They walked fifteen different paths through this area, collecting a massive amount of raw point cloud data that represented the real-world complexity of an outdoor space.

The first step in their work was to clean this raw data and teach a computer model to recognize the three main things they cared about: the ground, vegetation, and buildings. They manually labeled a portion of the data to serve as a guide, showing the computer what these three categories looked like. Once the model was trained, they tested its ability to identify these known objects across the entire dataset. The results were encouraging for the ground, which the model identified with very high accuracy. Vegetation was also recognized well, though the edges where trees met buildings sometimes caused confusion. Buildings proved the most difficult, likely because their shapes vary widely and the handheld scanner sometimes missed parts of walls or roofs due to the angle of the walk. Despite these imperfections, the model successfully learned to distinguish the three main categories across the complex terrain.

The true test, however, was what happened next. The researchers asked the model to look at the points it was not confident about and treat them as "unknown." Instead of forcing these uncertain points into the categories of ground, vegetation, or building, the system was designed to group them into new, separate clusters. This is the core of the novel class discovery approach: letting the computer find its own groups for things it doesn't understand. The model identified a small percentage of the total points as unknown, roughly nine percent of the data. It then sorted these unknown points into three new groups. When the researchers looked at what these new groups actually contained, they found a mix of success and limitation. One of the new groups did successfully isolate some distinct objects, such as utility poles and cables, which were indeed different from the main categories.

However, the study revealed that this method is not yet perfect for real-world outdoor data. A significant portion of the points placed into these new "unknown" groups actually belonged to the known categories. For instance, fragments of buildings or parts of trees that the model was unsure about were incorrectly grouped with the new objects. This happened because the initial model was not confident enough in its own predictions, causing it to mislabel parts of known objects as new discoveries. The researchers found that the quality of these new groups depended heavily on how well the model understood the original three categories. If the model struggled with a specific building or a patch of dense bushes, those errors rippled through the system, creating messy new groups that were not truly new objects, but rather confused pieces of old ones.

The study also highlighted how the physical nature of the data affected the results. Because the sensor was moving, the density of points changed depending on how fast the operator walked or how close they were to an object. In areas where the points were sparse or noisy, the model found it harder to tell the difference between a known object and a new one. Furthermore, the researchers had to decide on a fixed number of new groups to look for, and this rigid choice sometimes forced unrelated objects into the same bucket or split a single type of object into multiple groups. Without a perfect reference guide for what these new objects should be, the researchers relied on visual inspection to judge the results, confirming that while the system could find new patterns, those patterns were often unstable and mixed with errors from the known classes.

Ultimately, the research demonstrates that while the concept of novel class discovery holds great promise for mapping the real world, it faces significant hurdles when applied to data captured by moving sensors in complex environments. The system can identify that something is different, but it struggles to cleanly separate that difference from the noise and uncertainty of the data itself. The success of finding new classes is deeply tied to the accuracy of the initial recognition of known objects. If the computer is not sure about a building, it cannot be sure that a nearby pole is something new. The authors conclude that for this technology to become reliable for real-world use, the underlying models need to be more robust, the methods for spotting unknown objects need to be more flexible, and the evaluation of these new groups needs to be more systematic. The path forward involves refining how these systems learn from the messy, imperfect data of the real world, ensuring that when they discover something new, they are truly seeing something new, and not just a mistake.