LenghuSky-8: An 8-Year All-Sky Cloud Dataset with Star-Aware Masks and Alt-Az Calibration for Segmentation and Nowcasting

This paper introduces LenghuSky-8, an eight-year all-sky imaging dataset featuring star-aware masks and precise Alt-Az calibration that enables robust cloud segmentation and highlights the challenges of short-horizon cloud nowcasting for autonomous astronomical observatories.

The Gist

Imagine you are a space telescope sitting on a mountain in China, waiting to take pictures of the stars. But there's a problem: you can't see the stars if clouds are in the way. To know when to take a picture, you need a "weather forecaster" that looks at the sky every minute, 24/7, for years.

For a long time, scientists didn't have a good enough weather forecaster. They had short videos, or pictures that only worked during the day, or maps that didn't tell them exactly where in the sky the clouds were.

This paper introduces LenghuSky-8, a massive new tool that fixes all those problems. Here is the breakdown in simple terms:

1. The "Eight-Year Sky Diary" (The Dataset)

Think of this dataset as an 8-year diary of the sky, written by a camera that never sleeps.

• The Scale: It contains over 429,000 photos taken from 2018 to 2025.
• The Coverage: It's not just sunny days; it's 81% night-time photos, capturing everything from moonless darkness to full-moon brightness.
• The Location: It was taken at Lenghu, a mountain in China that is one of the best places on Earth for stargazing because the air is so dry and clear.

2. The "Smart Glasses" (The Masks)

Just looking at a photo isn't enough; you need to know exactly which pixels are clouds and which are stars.

• The Problem: Clouds are fuzzy. Sometimes a bright star looks like a cloud. Sometimes dust on the lens looks like a cloud.
• The Solution: The researchers created a "Smart Glasses" system. They used a super-smart AI (called DINOv3) to look at the photos and draw a map.
  • Blue areas: Clear sky (Go take a picture!).
  • Orange areas: Clouds (Wait!).
  • Pink areas: "Contamination" (This is dust, dew, or a weird light—ignore this part).
• The Result: They tested this on a small set of 1,111 photos and got it right 93% of the time, even when the moon was shining brightly.

3. The "GPS for the Sky" (Alt-Az Calibration)

This is the most magical part. Usually, a camera just gives you a picture. But for a telescope, you need to know: "That cloud is exactly 45 degrees up and 120 degrees to the left."

• The Analogy: Imagine looking at a fishbowl. The edges are stretched and warped. If you try to draw a map on a fishbowl, it gets distorted.
• The Fix: The researchers used the stars as a GPS grid. Because they know exactly where the stars should be, they can mathematically "un-warp" the fishbowl image.
• The Result: They can now tell a telescope, "Don't look there, there's a cloud," with an accuracy of less than half a degree. That's precise enough to tell a giant robot telescope exactly where to point.

4. The "Crystal Ball" (Nowcasting)

The researchers also asked: "If we see a cloud moving now, can we predict where it will be in 5 minutes?"

• The Test: They tried four different methods to guess the future:
  1. The Lazy Method: Just copy the last picture (assuming nothing changes).
  2. The Flow Method: Tracking how the clouds move like a river.
  3. The Memory Method (ConvLSTM): An AI that remembers how clouds usually move.
  4. The Dreamer Method (VideoGPT): A fancy AI that tries to "imagine" the next frame.
• The Surprise: The fancy AI (VideoGPT) actually did the worst. The "Lazy Method" (just copying the last frame) was almost as good as the complex ones.
• The Lesson: Clouds are chaotic. Predicting them 5 minutes into the future is incredibly hard, even for super-computers. The sky changes faster than our models can guess.

Why Does This Matter?

Imagine you are running a library of books (the stars). You only have a few minutes of daylight to read them before the sun comes up. If you waste time reading a page that is covered by a cloud, you miss a whole book.

LenghuSky-8 gives astronomers a perfect, 8-year history of the weather and a set of tools to:

1. Know instantly if the sky is clear.
2. Tell the robot telescope exactly where to point to avoid clouds.
3. Save money and time by not taking pictures of cloudy skies.

In short, this paper gives astronomers a super-powerful, 8-year weather map for the stars, helping them unlock the secrets of the universe without getting blocked by a little bit of rain.

Technical Summary

1. Problem Statement

Ground-based time-domain astronomical surveys (e.g., ZTF, Rubin Observatory, Tianyu Project) require real-time, site-scale awareness of cloud cover to optimize telescope scheduling and exposure plans. However, existing all-sky cloud datasets suffer from critical limitations:

• Short Duration: Most cover only months, preventing seasonal modeling.
• Daylight Bias: Few datasets include sufficient night-time data, which is crucial for astronomy.
• Lack of Calibration: They often lack precise astrometric calibration, making it difficult to map image pixels to specific altitude-azimuth (Alt-Az) coordinates for telescope pointing.
• Annotation Quality: Many rely on manual masking that does not scale or fail to handle complex scenarios like moonlight, dust, or dew.

The authors aim to bridge this gap by providing a long-term, geometrically calibrated, and rigorously annotated dataset suitable for both cloud segmentation and short-term nowcasting.

2. Methodology

A. Dataset Construction (LenghuSky-8)

• Source: Collected at the Lenghu astronomical site in Qinghai, China (a premier site with high altitude and dry climate) from 2018 to 2025.
• Scale: Contains 429,620 frames (512×512 resolution) with 81.2% night-time coverage.
• Acquisition: Captured using fisheye lenses (Sigma 4.5mm) on Canon DSLRs. The capture interval varies (5 min at night, 20 min during day) based on solar elevation.
• Data Split: Divided into two parts based on maintenance:
  • Part I (Pre-2023): Prone to lens dirt/dew but fewer obstructions.
  • Part II (Post-2023): Cleaner lenses but partially blocked by a nearby movable dome.

B. Annotation Strategy

• Classes: Three classes are defined: Sky (clear), Cloud (including thin clouds unsuitable for observation), and Contamination (snow, dew, dust, saturation, or artificial light).
• Manual Labeling: A conservative strategy was used for a 1,111-image reference set. Only high-confidence regions were labeled; ambiguous areas were marked as "contamination" or ignored to minimize human bias.
• Background Handling: A linear classifier based on DINOv3 CLS tokens was trained to automatically detect the state of a movable roof (raised/lowered) in Part II, enabling dynamic background masking.

C. Astrometric Calibration

• Challenge: Fisheye lenses introduce severe distortion, making standard tools like Astrometry.net ineffective for the full field of view.
• Solution: The authors adapted a method involving HEALPix patches.
  1. Stars are detected and grouped into HEALPix patches.
  2. Local coordinate systems are rotated to minimize distortion within each patch.
  3. Astrometry.net is applied to these local patches to derive World Coordinate Systems (WCS).
  4. For unresolved patches, a radial symmetric model is fitted to interpolate the Alt-Az coordinates.
• Accuracy: Achieved calibration uncertainties of ≈0.37° at the zenith and ≈1.34° at 30° altitude, sufficient for telescope scheduling.

D. Segmentation & Nowcasting Benchmarks

• Segmentation: Instead of training a heavy network from scratch, the authors utilized DINOv3 (ViT-L/16) pre-trained features. A linear probe was trained on local patch features to perform segmentation.
• Nowcasting: A benchmark was established to predict the next frame's logits (Sky/Cloud/Contamination) based on previous frames.
  • Baselines: Persistence (copying the last frame), Optical Flow (Farneback), ConvLSTM, and VideoGPT (VQ-VAE + Transformer).

3. Key Contributions

1. LenghuSky-8 Dataset: The first 8-year, all-sky, day-night dataset with star-aware masks and per-pixel Alt-Az calibration.
2. Robust Calibration Pipeline: A novel method to calibrate fisheye cameras using local HEALPatch-based astrometry, enabling integration with telescope schedulers.
3. Efficient Segmentation: Demonstration that a DINOv3 linear probe outperforms complex encoder-decoder architectures (U-Net, CloudSegNet) with significantly less training data and complexity.
4. Nowcasting Benchmark: A rigorous evaluation of cloud evolution prediction, highlighting the difficulty of short-term forecasting.

4. Results

Cloud Segmentation

• Performance: The DINOv3 linear probe (using only local features) achieved an overall accuracy of 93.3% ± 1.1% on the manually labeled test set.
• Comparison: It outperformed or matched more complex models:
  • DINOv3 Linear Probe: 93.3% Accuracy.
  • SegMAN (Large): 87.2% Accuracy.
  • U-Net: 85.1% Accuracy.
  • Encoder-Decoder: 80.4% Accuracy.
• Insight: Adding global tokens (CLS/Register) to the local features did not significantly improve performance, suggesting local spatial features are sufficient for cloud boundaries.

Weather Nowcasting

• Performance: All models struggled to significantly outperform the trivial baseline (persistence).
  • Trivial Baseline: 88.8% Accuracy.
  • ConvLSTM (Best): 89.0% Accuracy.
  • VideoGPT: Worst performance (87.0–87.2%).
• Insight: Cloud evolution at the site scale is highly stochastic in the short term (5–15 min). The marginal gain of ConvLSTM over persistence suggests that near-term cloud motion is difficult to model without physical constraints or higher-resolution meteorological data.

5. Significance and Future Work

• Autonomous Observatories: The dataset and Alt-Az maps enable the development of "cloud-aware schedulers" that can dynamically adjust telescope pointing to avoid clouds, maximizing scientific return for time-domain astronomy.
• Model Efficiency: The success of the DINOv3 linear probe suggests that large-scale self-supervised pre-training is highly effective for domain-specific tasks like cloud segmentation, reducing the need for massive labeled datasets.
• Open Science: The authors release the dataset, calibration tools, and an open-source toolkit to foster research in segmentation, nowcasting, and autonomous operations.
• Future Directions: The authors note that current nowcasting methods lack physical constraints. Future work should incorporate physical priors (e.g., advection consistency, mass continuity) and expand data collection to diverse sites to improve generalization.

In summary, LenghuSky-8 addresses a critical infrastructure gap in astronomy by providing a long-term, geometrically precise dataset that facilitates the transition from manual weather monitoring to automated, AI-driven observatory management.