TEMPO-VINE: A Multi-Temporal Sensor Fusion Dataset for Localization and Mapping in Vineyards

This paper introduces TEMPO-VINE, a comprehensive multi-temporal, multi-modal public dataset featuring heterogeneous sensors and ground truth trajectories across diverse vineyard conditions, designed to address the lack of realistic benchmarks for advancing autonomous localization, mapping, and place recognition in precision agriculture.

The Gist

Imagine you are trying to teach a robot to drive itself through a vineyard. Sounds simple, right? Just follow the rows of grapes. But here's the catch: a vineyard in February looks nothing like a vineyard in August.

In February, the vines are bare sticks, and the ground is short grass. By August, the vines are thick, leafy walls, and the grass might be knee-high. If you train a robot on the winter data, it will get hopelessly lost in the summer.

This is the problem the paper "TEMPO-VINE" is trying to solve.

The Problem: The "Video Game" Trap

Right now, most scientists test their self-driving farm robots in simulations (like a video game) or in very short, perfect field tests. It's like learning to drive a car only in an empty parking lot on a sunny day. You haven't learned how to handle rain, snow, or a sudden detour.

Because there wasn't a big, realistic "test track" for vineyards that changed with the seasons, robots struggled to work in the real world.

The Solution: The "Time-Traveling" Dataset

The authors created TEMPO-VINE, which is essentially a massive, multi-seasonal "training manual" for robots.

Think of it like a time-lapse movie of a vineyard. They didn't just take one photo; they drove a robot through the same rows of grapes 13 times over 10 months, from winter to late autumn.

• The Robot: They used a small, rugged rover (like a robot dog but with wheels) equipped with a "super-suit" of sensors.
• The Eyes: They gave the robot two different types of "3D eyes" (LiDARs):
  1. The Expensive Eye (Velodyne): A high-end, rotating laser scanner that sees everything clearly, like a luxury car's radar.
  2. The Budget Eye (Livox): A cheaper, newer laser scanner that sees things differently, like a budget-friendly security camera.
• The Other Senses: They also added a stereo camera (to see color and depth), a GPS (to know where it is on Earth), and an IMU (a gyroscope to know if it's tilting).

Why This is a Big Deal

The paper compares TEMPO-VINE to other datasets and finds it's the only one that does three specific things:

1. It changes with the seasons: It captures the "mood swings" of the vineyard (bare branches vs. leafy walls).
2. It has two different "eyes": It lets researchers test if a cheap robot can do the same job as an expensive one.
3. It has long rows: The vineyard rows are over 100 meters long, which is a real challenge for robots trying to stay on track.

What They Found (The "Report Card")

The authors tested popular robot navigation software on this new dataset to see how well it worked. Here is what they discovered:

• The "Winter" vs. "Summer" Shock: Algorithms that worked perfectly in March (when the vines were bare) often crashed or got lost in July (when the vines were thick). It's like trying to recognize a friend by their face, but then they put on a giant winter coat and a hat; suddenly, the robot doesn't know who they are.
• The "Budget" Eye Struggles: The cheaper LiDAR sensor had a harder time in the summer because its unique scanning pattern got confused by the dense leaves.
• The "Expensive" Eye is Reliable: The high-end sensor handled the changes better, but even it had trouble when the grass got too tall.
• Visuals are Tricky: Cameras (which rely on sight) were the worst at navigating because the lighting and shadows changed too much.

The Bottom Line

TEMPO-VINE is a gift to the robotics community. It's a realistic, messy, changing playground that forces robots to learn how to handle real life, not just a perfect simulation.

By making this data public, the authors hope to help engineers build robots that can actually work in vineyards year-round, regardless of whether it's winter, summer, raining, or sunny. It's the first step toward a future where robots can prune, harvest, and care for our grapes without needing a human to hold their hand.

Technical Summary

Here is a detailed technical summary of the paper "TEMPO-VINE: A Multi-Temporal Sensor Fusion Dataset for Localization and Mapping in Vineyards."

1. Problem Statement

The field of precision agriculture and agricultural robotics faces a significant bottleneck: the lack of realistic, standardized benchmarks for evaluating autonomous navigation systems.

• Limitations of Current Research: Most studies rely on controlled simulations or isolated field trials that fail to capture the dynamic variability of real-world agricultural environments (terrain, vegetation growth, weather).
• Specific Challenges in Vineyards: Vineyards present unique difficulties for Simultaneous Localization and Mapping (SLAM) and place recognition due to:
  • Dynamic Appearance: Rapid changes in vegetation structure (bare stems to full foliage) and ground cover (grass height) across seasons.
  • Structural Repetition: Long, repetitive rows cause ambiguity in place recognition.
  • Sensor Variability: Existing datasets often lack heterogeneous sensor suites (e.g., mixing high-end and low-cost LiDARs) or cover only short durations and small areas.
• The Gap: There is no public dataset that combines multi-seasonal data, heterogeneous LiDARs, and diverse vineyard architectures (trellis and pergola) with ground truth trajectories for robust algorithm evaluation.

2. Methodology and Dataset Construction

The authors introduced TEMPO-VINE, a large-scale, multi-modal, and multi-temporal dataset collected to bridge the gap between simulation and reality.

• Data Collection Platform:
  • Robot: A Husky A200 rover (ClearPath Robotics) teleoperated manually to simulate realistic autonomous driving conditions (including drifts and off-center row traversal).
  • Sensors: A heterogeneous suite including:
    • 3D LiDARs: A high-end Velodyne VLP-16 (rotating, 16 channels) and a low-cost Livox Mid-360 (single-beam, non-repetitive scanning pattern).
    • Vision: Intel Realsense D435 RGB-D camera.
    • Localization: RTK-GNSS (Swift Navigation Duro) and AHRS (MicroStrain 3DM-GX5) for ground truth.
• Collection Campaigns:
  • Duration: 10 months (February to November 2025), covering Winter, Spring, Summer, and Autumn.
  • Locations: Two distinct vineyard types in Agliè, Italy:
    1. Trellis Guyot: 110m x 25m, 10 rows.
    2. Pergola: 110m x 15m, 4 rows.
  • Trajectories: Multiple runs per campaign, including revisits to the same rows to enable loop-closure and place recognition testing.
• Dataset Organization:
  • Data is provided in ROS-compatible format (.bag files using .mcap.zstd compression).
  • Includes raw sensor streams, metadata, RGB video, and ground truth trajectories (local ENU and geo-referenced).
  • Total length: 38 km of data across 13 campaigns.

3. Key Contributions

1. First Multi-Seasonal Vineyard Dataset: TEMPO-VINE is the first public dataset capturing the full lifecycle of vine growth (from bare stems to fruit-bearing) across two different architectural styles (trellis and pergola).
2. Heterogeneous LiDAR Benchmark: It uniquely provides data from both a standard rotating LiDAR (Velodyne) and a non-repetitive scanning LiDAR (Livox), enabling research into cost-effective vs. high-performance robotic platforms.
3. Comprehensive Ground Truth: High-precision RTK-GNSS and AHRS data provide accurate ground truth trajectories for evaluating SLAM and localization algorithms.
4. Realistic Complexity: The dataset includes natural variations in weather, grass height, and vegetation density, creating "domain gaps" that challenge current machine learning and perception algorithms.
5. Open Access: The dataset, processing tools, and benchmarking results are publicly available with a standard file structure for easy integration.

4. Experimental Results and Evaluation

The authors evaluated state-of-the-art algorithms on the dataset to demonstrate its difficulty and utility.

• SLAM Performance:
  • Algorithms Tested: RTAB-Map (RGB-D), Fast-LIO (LiDAR-Inertial), and LIO-SAM.
  • Findings:
    • LIO-SAM (using Velodyne) showed the most robust performance across all seasons.
    • Fast-LIO performed well with Livox data but struggled in summer trellis scenarios due to heavy vegetation occlusion.
    • RTAB-Map consistently underperformed, highlighting the difficulty of RGB-D-based SLAM in repetitive, texture-poor agricultural rows.
    • Seasonal Impact: Algorithms often failed (ATE > 4m) when transitioning between seasons (e.g., Winter to Summer) due to drastic structural changes.
• Place Recognition:
  • Methods Tested: Handcrafted descriptor (Scan Context) vs. Deep Learning descriptors (PointNetVLAD, MinkLoc3Dv2).
  • Findings:
    • Scan Context outperformed deep learning methods, which were originally trained on urban datasets and struggled to generalize to agricultural environments.
    • Seasonal Generalization: Performance dropped significantly when querying across different growth stages (e.g., querying a summer map with a winter database).
    • Sensor Comparison: The Velodyne LiDAR generally provided better recall than the Livox, though both suffered from seasonal variability.

5. Significance and Impact

• Advancing Agricultural Robotics: TEMPO-VINE provides the necessary infrastructure to move beyond simulation, allowing researchers to develop and validate robust autonomous systems capable of operating in real, dynamic vineyards.
• Cost-Effective Automation: By including low-cost LiDAR data, the dataset supports the development of affordable robotic solutions suitable for small-scale farms, a critical factor for global adoption.
• Benchmarking Standard: It establishes a new standard for evaluating sensor fusion and SLAM in unstructured, seasonal environments, highlighting the specific need for algorithms that can handle long-term temporal variability.
• Future Directions: The authors plan to extend the dataset to two years of coverage and enrich the benchmarking suite with introspection analyses to further guide algorithm development.

In conclusion, TEMPO-VINE addresses a critical void in agricultural robotics research by providing a challenging, realistic, and diverse dataset that forces algorithms to confront the complexities of seasonal change and structural repetition, thereby accelerating the deployment of reliable autonomous farming systems.