FPGA-Enabled Machine Learning Applications in Earth Observation: A Systematic Review

This systematic review analyzes 68 experiments deploying machine learning models on FPGAs for Earth Observation, introducing dual taxonomies for model architectures and implementation strategies to address the challenges of onboard processing in the NewSpace era.

The Gist

Imagine you are the captain of a tiny, fast-moving spaceship (a satellite) or a high-tech drone flying over the Earth. Your job is to take millions of photos of our planet to track forests, find ships, or monitor clouds.

The Problem:
These cameras are so powerful they generate a massive flood of data. But your spaceship has a tiny, slow radio (bandwidth) to send that data back to Earth. If you try to send everything, the radio gets clogged, and you miss important updates. Plus, waiting for Earth to tell you what to do takes too long. You need to make decisions right now, while you are still flying.

The Solution:
You need a brain on board that is smart enough to look at the photos, figure out what's important, and only send the good stuff back. This is where Machine Learning (AI) comes in. But AI usually needs huge, power-hungry computers (like the ones in data centers) to run. Your spaceship can't carry a supercomputer; it has strict limits on weight, power, and size.

The Hero: The FPGA
Enter the FPGA (Field-Programmable Gate Array). Think of an FPGA not as a standard computer chip, but as a giant, magical Lego board.

• A standard computer (CPU) is like a pre-built kitchen: it has a stove, a fridge, and a sink. It can cook almost anything, but it's not the most efficient if you just want to boil water.
• An FPGA is a box of loose Lego bricks. You can snap them together to build exactly what you need. If you need a super-fast "Cloud Detector," you build a machine specifically for that. If you need a "Ship Finder," you rebuild the machine for that.
• Because you build the machine specifically for the job, it's incredibly fast and uses very little power. It's like building a custom race car instead of driving a family SUV.

What This Paper Did:
The authors, Cédric, Dirk, and Martin, acted like detectives. They looked at 68 different experiments where scientists tried to put these "Lego brains" (FPGAs) on satellites and drones to run AI. They wanted to answer: Is this actually working? What are we building? And how do we make it better?

Here are the key takeaways, explained simply:

1. What are they building? (The Tasks)

Most of the time, these "Lego brains" are doing three main things:

• Surveillance: "Is that a ship? Is that a tank? Is that a drone?" (Like a security guard scanning a crowd).
• Mapping: "Is this area a forest, a city, or a desert?" (Like sorting a pile of mixed-up toys into boxes).
• Weather: "Are there clouds blocking the view?" (Like a bouncer checking if the sky is clear).

2. The Brains They Use (The Models)

The scientists are mostly using Convolutional Neural Networks (CNNs). Think of these as a team of specialized detectives.

• The "Custom" Detectives: Most researchers didn't use off-the-shelf models. They built their own, smaller, custom detectives because the spaceship doesn't have room for the big, famous ones (like the ones used in self-driving cars).
• The "Tiny" Detectives: They are using "Lite" versions of famous models (like MobileNet or YOLO-tiny). It's like taking a full-sized library and shrinking it down to a pocket-sized guidebook that still has all the answers.

3. How they make it fit (The Optimization)

Since the "Lego board" is small, they have to be clever to fit the AI on it. They use three main tricks:

• Quantization (The Shrink Ray): Standard AI uses very precise numbers (like 3.1415926). The FPGA doesn't need that much precision. They round the numbers down to simple integers (like 3). It's like measuring a room with a ruler instead of a laser scanner. You lose a tiny bit of detail, but it fits in your pocket and works 10x faster.
• Pruning (The Edit): They cut out the parts of the AI that aren't doing much work. It's like editing a movie to cut out boring scenes so it runs faster.
• Lightweight Backbones: They use simpler structures for the AI, like a skeleton instead of a full body.

4. The Tools (The Frameworks)

Building these Lego brains is hard.

• Manual Building: Some scientists build the circuit by hand, brick by brick (using code called HDL). This gives them the most control but takes forever.
• Automatic Builders: Others use software tools (like Vitis AI or FINN) that act like a "3D printer" for circuits. You give it the design, and it prints the circuit for you. It's faster, but sometimes the result isn't quite as efficient as a hand-built one.

5. The Results & The Future

• Success: They found that FPGAs are winning. They are much more power-efficient than standard computers and can handle the radiation of space better than most other chips.
• The Gap: However, there are still missing pieces.
  • Trust: We need to know why the AI made a decision. If the satellite says "Fire detected," we need to know it's not a glitch.
  • New Models: They mostly use older AI styles. Newer, cooler AI models (like Transformers) are barely being used on these tiny chips yet because they are too heavy.
  • Sharing: Many scientists don't share their code or data, making it hard for others to learn from their mistakes.

The Big Picture

This paper is a roadmap. It tells us that putting AI brains on tiny satellites is no longer science fiction; it's happening. By using these reconfigurable "Lego" chips, we can turn our satellites from simple cameras into smart, autonomous observers that can spot a fire, track a storm, or find a ship the moment it happens, without waiting for a signal from Earth.

It's the difference between sending a photo to a friend and asking them what they see, versus having a friend who lives inside the camera and just texts you: "Hey, I see a fire. Send help."

Technical Summary

Here is a detailed technical summary of the paper "FPGA-Enabled Machine Learning Applications in Earth Observation: A Systematic Review" by Cédric Léonard, Dirk Stober, and Martin Schulz.

1. Problem Statement

The Earth Observation (EO) sector is undergoing a paradigm shift driven by the "NewSpace" era and the proliferation of small satellites (CubeSats, SmallSats) and Unmanned Aerial Vehicles (UAVs). While these platforms generate massive volumes of high-resolution data, they face severe constraints:

• Bandwidth Limitations: Downlink capacity is stagnant while data generation grows exponentially, creating a bottleneck.
• Resource Constraints: Small platforms have strict Size, Weight, Power, and Cost (SWaP-C) limits, restricting onboard compute capacity.
• Latency Requirements: Many missions (e.g., disaster response, target surveillance) require real-time decision-making, making ground-based processing infeasible.

Current solutions often rely on Cloud or Ground processing, leading to data loss or delays. While Machine Learning (ML) offers the capability for autonomous, onboard processing, standard hardware (CPUs, GPUs) is often too power-hungry or lacks the necessary radiation hardening for space. Field-Programmable Gate Arrays (FPGAs) offer a unique balance of reconfigurability, power efficiency, and radiation tolerance, yet there is no comprehensive synthesis of how ML models are currently deployed on FPGAs for Remote Sensing (RS).

2. Methodology

The authors conducted a Systematic Literature Review (SLR) adhering to the PRISMA 2020 guidelines to ensure transparency and reproducibility.

• Search Strategy: A query was executed on the Web of Science database covering the period 2014–2024. The search terms combined ML/DL concepts, Earth Observation/Remote Sensing, and FPGA hardware.
• Selection Criteria:
  • Inclusion: Experimental studies deploying ML models on FPGAs for RS applications (Spaceborne or Airborne).
  • Exclusion: Review papers without experiments, non-ML studies, non-RS applications (e.g., autonomous driving), and studies lacking FPGA implementation details.
• Dataset: From an initial pool of 119 records, 48 studies (comprising 68 distinct experiments) were selected for deep analysis.
• Analysis Framework: The authors developed two specialized taxonomies to categorize the literature:
  1. RS/ML Taxonomy: Categorizing applications by task (Classification, Segmentation, Detection), data modality (RGB, Hyperspectral, SAR), and model architecture.
  2. FPGA Implementation Taxonomy: Categorizing design strategies, frameworks (Manual vs. Automatic), optimization techniques, and performance metrics.

3. Key Contributions

The paper makes five primary contributions to the field:

1. Systematic Analysis: A rigorous, transparent review of 68 experiments following PRISMA guidelines, filling a gap where no prior work covered the intersection of RS, ML, and FPGAs.
2. Dual Taxonomies:
   • A taxonomy mapping RS applications to ML problem formulations (e.g., distinguishing between pixel-level and patch-level segmentation).
   • A taxonomy classifying FPGA implementation strategies (Manual HDL/HLS vs. Automatic frameworks like Vitis AI and FINN).
3. Research Questions (RQs) Addressed: The review answers eight specific RQs covering the research landscape, model diversity, hardware motivations, optimization strategies, design frameworks, performance trade-offs, and future synergies.
4. Identification of Research Gaps: The authors pinpoint critical missing areas, including the lack of Transformer (ViT) and Recurrent (RNN) implementations, underutilization of Knowledge Distillation, and a scarcity of Uncertainty Quantification (UQ) and Explainable AI (xAI) in onboard systems.
5. Open Resources: The authors released a public repository containing all extracted data, code, and the PRISMA flow diagram to facilitate future benchmarking and reproducibility.

4. Key Results and Findings

A. Applications and Models (RQ1 & RQ2)

• Dominant Tasks: Target Surveillance (44% of studies) and Landscape/Environment Monitoring (40%) are the primary focus.
• ML Tasks: Classification is the most common formulation (35%), followed by Object Detection. Regression is severely underrepresented (<5%).
• Model Architectures: Convolutional Neural Networks (CNNs) dominate (74% of models).
  • Custom CNNs (27%) are more prevalent than standard backbones (ResNet, VGG), likely due to the need for medium-sized, hardware-tailored architectures (6–10 layers).
  • Single-shot detectors (YOLO, SSD) are the standard for object detection; two-stage detectors (R-CNN) are absent due to computational cost.
  • Transformers (ViT) and RNNs are virtually non-existent in the surveyed FPGA literature, representing a significant gap.

B. Hardware and Platform (RQ3)

• Platform Dominance: AMD (Xilinx) FPGAs account for 96% of the hardware used.
• Preferred Devices: Zynq SoCs (57%) are the most common, integrating ARM processors with FPGA logic, ideal for edge tasks.
• Motivation: FPGAs are chosen over CPUs/GPUs primarily for power efficiency (often 10x–30x better GOP/s/W) and radiation tolerance (critical for space). While GPUs offer higher peak performance, their power consumption makes them unsuitable for most small satellite payloads.

C. Optimization and Design Strategies (RQ4, RQ5, RQ6)

• Optimization Techniques:
  • Quantization: 89% of Deep Learning experiments use quantization, predominantly to 8-bit integer (i8). Mixed-precision is emerging but less common.
  • Lightweight Backbones: 24% of models use efficient architectures like MobileNet or GhostNet.
  • Pruning: 26% of studies employ pruning to reduce model size.
• Implementation Frameworks:
  • Manual Design (HDL/HLS): ~45% of studies, offering fine-grained control but high development effort.
  • Automatic Frameworks: Vitis AI and FINN are the leading tools. Vitis AI supports flexible designs (loading weights from off-chip), while FINN specializes in binarized networks with on-chip weights.
• Performance:
  • Throughput: Ranges from <1 GOP/s to >3 TOP/s (using All-Adder networks).
  • Efficiency: High-end designs achieve >300 GOP/s/W.
  • Latency: Real-time performance (e.g., <100ms for cloud detection) is frequently achieved, though often at the cost of accuracy or aggressive quantization.

D. Synergies and Future Missions (RQ7 & RQ8)

• Scalability: Larger FPGAs (High-End) allow for larger models with less aggressive compression, but resource utilization efficiency is a better predictor of performance than raw device size.
• Real-World Deployments: The paper highlights successful missions like PhiSat-1 (CloudScout model for cloud masking) and OPS-SAT, proving that FPGA-based AI is operationally viable for reducing downlink data by filtering cloudy images.
• UAVs: UAVs benefit from real-time obstacle detection and terrain classification, often using lightweight CNNs on Zynq-7000/US+ platforms.

5. Significance and Future Directions

This survey establishes a foundational benchmark for the intersection of AI and hardware in Earth Observation. Its significance lies in:

• Validating Edge AI: It confirms that FPGAs are the current standard for deploying ML on resource-constrained space and aerial platforms.
• Highlighting Gaps: It identifies that the field is lagging behind in adopting modern architectures (Transformers) and advanced ML concepts (Uncertainty Quantification, Knowledge Distillation).
• Guiding Future Research: The authors recommend:
  • Developing FPGA-aware Neural Architecture Search (NAS).
  • Integrating Explainable AI (xAI) and Uncertainty Quantification (UQ) to build trust in autonomous onboard decisions.
  • Exploring Coarse-Grained Reconfigurable Arrays (CGRAs) (e.g., AMD Versal AI Engines) as a potential successor to traditional FPGAs for specific workloads.
  • Improving reporting standards (memory usage, power, bit-widths) to enable fair comparisons.

In conclusion, the paper argues that while significant progress has been made in deploying CNNs on FPGAs for RS, the field must evolve to support more complex models, ensure reliability through interpretability, and leverage emerging hardware architectures to fully realize the potential of autonomous Earth Observation.