VLN-AVP: Zero-Shot Vision-Language Navigation with Hybrid Long-Short-Term Memory for Autonomous Valet Parking
The paper proposes VLN-AVP, a zero-shot navigation framework for autonomous valet parking that combines Bird's-Eye-View perception with Vision-Language Models and a hybrid memory system to eliminate map dependency, interpret natural language instructions, and achieve superior performance in both simulation and real-world underground parking environments.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are teaching a robot how to drive. For a long time, the only way to teach a robot to park a car was to give it a perfect, pre-drawn map of the entire parking garage, like a treasure map with every turn and obstacle marked in ink. This works great if you know the building, but if you walk into a new, unfamiliar garage, the robot is stuck because it doesn't have its map. This is the world of "Autonomous Valet Parking" (AVP), where cars park themselves for you. But what if the robot could just look around, read the signs, and understand a human saying, "Find a spot near the elevator," without needing a map at all? This is where "Vision-Language Navigation" (VLN) comes in. Think of VLN as teaching a robot to understand the world through a combination of its eyes (vision) and its brain's ability to read and reason with words (language). The big question researchers are asking is: Can we make a self-driving car smart enough to navigate a strange, underground parking lot just by listening to us and looking at the signs, without ever seeing a pre-built map before?
This paper introduces a new system called VLN-AVP, which attempts to solve exactly that problem. The authors propose a "zero-shot" navigation framework, which is a fancy way of saying the system can handle a brand-new parking garage it has never seen before, using only natural language instructions from a human. Instead of relying on a pre-built map, the system uses a powerful "Vision-Language Model" (VLM)—think of it as a super-smart robot brain that can look at a picture and read a sentence to figure out what to do. However, the authors found that just giving the robot a smart brain isn't enough; it needs a better memory to avoid getting confused.
To fix this, the team built a hybrid memory system with two distinct parts. First, there is a Short-Term Memory, which acts like a robot's immediate "working memory." Since the smart brain (the VLM) doesn't look at the world very fast, it might miss a sign that flashes by quickly. The Short-Term Memory keeps a running list of recent signs and visual clues, so the robot doesn't forget that it just saw an "Exit" sign three seconds ago. Second, there is a Long-Term Topological Memory, which is like a mental sketch the robot draws of the garage as it drives. Every time the robot successfully finds a path or a landmark (like a specific elevator), it adds it to this sketch. If the robot has to navigate the same garage again, it can use this sketch to move faster and more efficiently, rather than asking the smart brain to figure everything out from scratch every time.
The researchers tested this idea in two ways: in a high-fidelity computer simulation and on a real car in a real underground parking garage. They created a new dataset called VLN-AVP, featuring 10 different high-quality 3D parking scenes and over 1,000 navigation episodes, which is the largest collection of underground garage scenes ever made for this type of research.
The results were promising. In the simulations, the VLN-AVP system was significantly better than other methods. It achieved a success rate that was over 25% higher than other Vision-Language Navigation methods and over 15% higher than other autonomous driving methods. In the real-world tests with the actual car, the system also performed well, successfully navigating complex instructions like "find a spot near the elevator hall" and even correcting its course when a human told it, "No, that's the wrong elevator, keep going." The study suggests that by combining a smart language brain with a clever two-part memory system, we can make self-parking cars that are much more flexible and don't need a pre-drawn map to get the job done.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.