Satellite-Based Detection of Looted Archaeological Sites Using Machine Learning

This paper presents a scalable satellite-based pipeline using PlanetScope imagery and machine learning to detect looted archaeological sites in Afghanistan, demonstrating that ImageNet-pretrained CNNs with spatial masking significantly outperform traditional machine learning and geospatial foundation models in identifying localized looting signatures.

The Gist

Imagine a vast, ancient library where every book is a piece of human history buried in the ground. Now, imagine a thief sneaking in, not to steal the books, but to dig holes in the floor, tearing up the pages and scattering the dust. This is archaeological looting. It's a silent crime that destroys our shared heritage, often happening in remote, dangerous places where no human guard can watch 24/7.

This paper is about building a digital security guard that never sleeps, using satellites to spot these thieves from space.

Here is the story of how they did it, explained simply:

1. The Mission: Watching the Unwatchable

The researchers focused on Afghanistan, a country rich in history but plagued by looting. They had a list of nearly 2,000 ancient sites. Some were safe (preserved), and some had been ransacked (looted).

• The Problem: You can't send a human to check every single site every month. It's too expensive and dangerous.
• The Solution: They used PlanetScope satellites. Think of these as high-tech cameras in the sky that take a picture of the same spot once a month, like a time-lapse video of the Earth.

2. The Detective Tools: Two Different Approaches

The team tried two different ways to teach a computer to spot the "holes" in the library floor.

Approach A: The "Art Student" (Deep Learning/CNNs)

Imagine showing a student thousands of photos of "clean" floors and "messy" floors. You don't tell them what to look for (like "look for a square hole"); you just let them study the pictures until they figure it out themselves.

• The Trick: They used a pre-trained "Art Student" (a model called ResNet) that had already learned to recognize cats, cars, and trees from millions of internet photos. They then asked this student to learn the difference between looted and safe sites.
• The Secret Weapon (Spatial Masking): This is the most important part. Imagine looking at a messy room. If you can only see the messy rug and ignore the clean walls and the window, it's much easier to spot the mess.
  • The researchers drew a digital "mask" (a stencil) over the exact shape of the ancient site.
  • They told the computer: "Ignore the roads, the farms, and the modern houses nearby. Only look at the ancient dirt."
  • Result: This "Art Student" became incredibly smart, getting it right 92.6% of the time.

Approach B: The "Data Analyst" (Traditional Machine Learning)

This approach is like hiring a detective who doesn't look at the whole picture but instead measures specific clues.

• They manually told the computer what to measure: "How green is the grass?" "How rough is the soil texture?" "Is the color weird?"
• They also tried using "Foundation Models" (super-smart AI models trained on all of Earth's data) to get a quick summary of the image.
• Result: These methods were good, but not great. The best one got it right only 71% of the time. It was like trying to find a needle in a haystack by measuring the weight of the hay, rather than just looking for the needle.

3. The Big Surprises

The study found some fascinating things:

• Context is King: The "Art Student" (Deep Learning) crushed the "Data Analyst" (Traditional ML). This suggests that looting leaves a very subtle, complex pattern that is hard to describe with simple rules but easy for a smart AI to "see" if it focuses on the right spot.
• The Mask Matters: Removing the background noise (roads, farms) was the single biggest factor in success. It boosted the AI's performance by nearly 45%. It's like wearing noise-canceling headphones to hear a whisper.
• Time Travel: They found that training the AI on just one year of data (2023) worked better than feeding it 7 years of data. Why? Because looting is a slow process. A site might look safe in 2016 but look destroyed in 2023. Mixing them up confused the AI. It's better to judge the site based on its current state.

4. What Does This Mean for the Future?

This isn't just about math; it's about saving history.

• Scalability: This system can check thousands of sites in minutes, something humans could never do.
• Actionable Intelligence: By knowing exactly where the looting is happening, authorities can send guards, alert local communities, or stop the sale of stolen artifacts.
• The "Library" Analogy: If the world's history is a library, this technology is the new security system that spots the holes in the floor before the books are gone forever.

In a nutshell: The researchers built a super-smart satellite detective that ignores the background noise and focuses entirely on the ancient ground. By using a "pre-trained" AI and a digital stencil to zoom in on the site, they created a tool that is far better at spotting looting than any previous method. It's a powerful step toward protecting our past for the future.

Technical Summary

1. Problem Statement

The looting of archaeological sites poses a critical threat to global cultural heritage, particularly in remote or conflict-affected regions like the Middle East and North Africa. Traditional monitoring methods (ground-based or manual visual inspection of satellite imagery) are labor-intensive, costly, and lack scalability.

• Challenge: Looting creates subtle, spatially diffuse surface alterations (disturbed soil texture, irregular micro-relief, spectral anomalies) that are often visually ambiguous and overlap with natural processes like erosion or agriculture.
• Goal: To develop a scalable, automated pipeline using medium-resolution satellite imagery to detect looted archaeological sites across large geographic areas.

2. Dataset

The authors constructed the largest dataset of its kind to date:

• Scope: 1,943 archaeological sites in Afghanistan (898 looted, 1,045 preserved).
• Source: Data derived from the Archaeological Gazetteer of Afghanistan, verified by expert archaeologists using high-resolution imagery (Google Earth, ESRI, Bing).
• Imagery: PlanetScope monthly mosaics (RGB + NIR) at 4.7m/pixel resolution, spanning January 2016 to December 2023 (96 months).
• Annotations:
  • Labels: Binary classification (Looted vs. Preserved) based on the state of the site as of December 2023.
  • Spatial Masks: Manually annotated polygon footprints rasterized into binary masks. These masks focus the model on the site area, suppressing non-site context (roads, settlements, fields).
  • Note: The dataset acknowledges "temporal label noise," as sites labeled "looted" in 2023 may appear intact in earlier years (2016–2020) because looting is a cumulative process.

3. Methodology

The study compares two primary methodological families: End-to-End Deep Learning (CNNs) and Traditional Machine Learning (ML) using engineered features and foundation model embeddings.

A. Handcrafted Features (Traditional ML)

The authors extracted 42-dimensional feature vectors per monthly composite, including:

• Spectral Indices: Vegetation and moisture indices, plus disturbance-sensitive indices.
• Texture Features:
  • GLCM (Gray-Level Co-occurrence Matrix): Capturing contrast, homogeneity, energy, and entropy.
  • LBP (Local Binary Pattern): Encoding micro-texture via local thresholding.
• Statistics: Mean, standard deviation, and temporal slopes across multi-month windows.

B. Foundation Model Embeddings

Embeddings were extracted from monthly mosaics using six pre-trained geospatial foundation models:

• DINOv3 (ViT-L/16)
• GeoRSCLIP (ViT-B/32)
• Prithvi-EO 2.0 (ViT-Huge/14)
• SatCLIP (Location + Vision embeddings combined)
• Satlas-Pretrain (ResNet-152)
• SatMAE (Masked Autoencoder)

C. Models and Aggregation

• CNNs: ResNet (18, 34, 50) and EfficientNet (B0, B1) trained end-to-end on raw RGB patches.
• Traditional ML: Logistic Regression, Random Forest (RF), Gradient Boosting (GB), and XGBoost (XGB).
• Temporal Aggregation: To handle the 8-year time series, features were aggregated using Mean, Median, Concatenation, or PCA compression.
• Spatial Masking: A critical preprocessing step where a binary mask is applied to the image patch (element-wise multiplication) to isolate the archaeological footprint before feature extraction or CNN inference.

4. Key Contributions

1. Comprehensive Framework: The first systematic comparison between end-to-end CNNs and traditional ML (handcrafted features + foundation model embeddings) for archaeological looting detection.
2. Temporal Consistency Insight: Demonstrated that training on a single year of imagery (2023) yields higher accuracy than multi-year training, as it mitigates temporal label noise caused by the gradual nature of looting.
3. Quantified Benefits of Pretraining and Masking:
   • ImageNet Pretraining: Provided a 6–14% F1 score improvement over random initialization, even with domain shift.
   • Spatial Masking: Resulted in a massive 30–45% F1 score improvement by suppressing distracting background context.
4. Dataset Release: The release of the largest public dataset for this task (1,943 sites) with multi-year imagery and spatial masks.

5. Experimental Results

The experiments were conducted on a 70/10/20 train/validation/test split with 5-fold cross-validation.

• Best Overall Performance:
  • ResNet-50 (ImageNet pre-trained + Spatial Masking) achieved the highest F1 score of 0.926 (Accuracy: 0.930).
  • This significantly outperformed the best traditional ML setup.
• Traditional ML Performance:
  • The strongest traditional setup (SatCLIP-V + Random Forest + Mean aggregation) achieved an F1 score of 0.710.
  • Handcrafted features combined with XGBoost and PCA achieved an F1 of 0.690.
• Impact of Spatial Masking:
  • For CNNs, masking improved F1 by +0.30 to +0.45.
  • For Traditional ML, masking improved handcrafted features and SatCLIP but degraded performance for some foundation models (e.g., DINOv3, SatMAE), suggesting these models rely on broader scene context.
• Feature Importance (SHAP Analysis):
  • Texture-based features were most discriminative.
  • The top feature was Sobel edge strength on the NIR band (SHAP=0.386), effectively detecting excavation boundaries.
  • GLCM texture measures (contrast, entropy, homogeneity) dominated the top 10 features.

6. Significance and Conclusion

• Superiority of End-to-End Learning: The study concludes that for subtle, localized disturbances like looting, end-to-end CNNs with transfer learning (ImageNet) and spatial masking are superior to traditional ML pipelines.
• Operational Viability: The high F1 score (0.926) demonstrates that automated monitoring is feasible at scale, offering a practical tool for heritage protection agencies.
• Role of Spatial Priors: Explicit spatial masking acts as a powerful attention mechanism, forcing models to focus on relevant disturbance patterns rather than background noise.
• Future Directions: The authors propose scaling this framework to other regions (Syria, Sudan, Egypt) and integrating active learning to reduce reliance on expert annotations.

Repository: The code and dataset are available at https://github.com/microsoft/looted_site_detection.