Language Conditioning Improves Accuracy of Aircraft Goal Prediction in Non-Towered Airspace

This paper presents a multimodal framework that integrates natural language understanding of pilot radio calls with trajectory data to significantly improve the accuracy of aircraft goal prediction in non-towered airspace compared to motion-only baselines.

The Gist

Imagine you are driving a self-driving car in a busy neighborhood where there are no traffic lights, no police officers, and no stop signs. Instead, the drivers rely on a shared radio frequency to talk to each other. One driver says, "I'm turning left at the next block," and another says, "I'm heading straight to the grocery store."

Now, imagine that self-driving car has to guess where the other cars are going. If it only looks at where the cars were in the last few seconds (their speed and direction), it might guess wrong. But if it can listen to what the drivers are saying on the radio, it can guess much better.

This paper is about teaching an autonomous airplane to do exactly that in the sky.

The Problem: The "Silent" Sky

Most small airports in the US don't have control towers. There's no one telling the planes where to go. Instead, human pilots talk to each other over a radio (called CTAF) to avoid crashing. They say things like, "Skyhawk 53X, entering the pattern for Runway 8."

For a human pilot, this is easy to understand. But for a robot plane, it's a nightmare. Current robot planes are like drivers who only look at the road ahead but have their ears plugged. They can see a plane moving, but they don't know why it's moving or where it intends to land. They have to guess based only on physics, which is often wrong.

The Solution: Giving the Robot "Ears"

The researchers built a system that lets the robot plane listen to the radio, understand the words, and use that information to predict the future. They call this "Language Conditioning."

Think of it like this:

• Old Way (Motion Only): You see a person walking toward a door. You guess they are going inside. But maybe they are just stretching their legs! You might be wrong.
• New Way (Motion + Language): You see the person walking toward the door, and you hear them say, "I'm going to get a coffee." Now you know exactly what they are doing. Your prediction is much more accurate.

How It Works (The "Brain" of the Plane)

The researchers created a three-step process for their robot plane:

1. The Translator (Speech-to-Text):
   First, the plane listens to the static-filled radio chatter. It uses advanced AI (like a super-smart Siri) to turn that garbled audio into clear text.

   • The Trick: They taught the AI specific "airport slang" (like "downwind," "base," "runway 8") so it doesn't get confused by the jargon.

2. The Interpreter (The "What do they mean?" Step):
   Once the text is written down, a Large Language Model (like a very smart chatbot) reads it and figures out the intent.

   • Instead of keeping the whole sentence, it boils it down to a simple label: "Landing on Runway 8" or "Taking off to the West."

3. The Predictor (The Crystal Ball):
   Finally, the robot combines two pieces of information:

   • Where the plane is going right now (its physical path).
   • What the pilot said they want to do (the intent label).

   It feeds these into a math model that spits out a probability map. It says, "There is a 90% chance the plane will land on Runway 8, and a 10% chance it will go somewhere else."

The Results: Listening Saves Lives

The team tested this on real data from a busy regional airport. They compared their "listening" robot against robots that only "watched."

• The "Watch-Only" Robot: Made mistakes often because it didn't know the pilot's plan. It was like guessing where a driver is going just by looking at their steering wheel.
• The "Listening" Robot: Made far fewer mistakes. By knowing the pilot's intent, it could predict the landing spot much more accurately.

In fact, when they tried to trick the listening robot by hiding the radio words (making it act like it couldn't hear), its performance dropped significantly. This proved that hearing is just as important as seeing for safe flying.

Why This Matters

This isn't just about making planes smarter; it's about safety.

• The 1,500-Foot Rule: Safety rules say planes must stay at least 1,500 feet apart. If a robot plane guesses wrong about where another plane is going, it might get too close.
• The Future: As more autonomous planes enter the sky, they need to be able to "speak the language" of human pilots to avoid crashes. This paper shows that if you teach a robot to listen, it becomes a much safer neighbor in the sky.

In a nutshell: This paper teaches self-flying planes to stop guessing and start listening. By understanding human radio calls, these robots can predict where other planes are going with much higher accuracy, making the skies safer for everyone.

Technical Summary

Here is a detailed technical summary of the paper "Language Conditioning Improves Accuracy of Aircraft Goal Prediction in Non-Towered Airspace."

1. Problem Statement

The paper addresses the challenge of safe autonomous aircraft operation in non-towered airspace (airports without control towers), which constitutes 92% of US airports and 90% globally.

• The Core Issue: In these environments, pilots coordinate via unstructured voice radio calls on the Common Traffic Advisory Frequency (CTAF) rather than relying on centralized air traffic control. Autonomous aircraft currently rely solely on observed trajectories (ADS-B data) and historical motion patterns to predict the future goals (intent) of other aircraft.
• The Gap: Current state-of-the-art trajectory prediction methods ignore radio communications. Consequently, autonomous systems miss critical coordination information, leading to higher prediction errors and potential safety risks in mixed human-autonomy operations.
• Research Question: Does incorporating natural language radio call conditioning improve an autonomous aircraft's ability to predict the motion goals of other aircraft compared to trajectory-only baselines?

2. Methodology

The authors propose a multimodal framework that fuses natural language understanding with spatial reasoning. The pipeline consists of three main stages:

A. Context-Enhanced Speech-to-Text (STT) & Speaker Identification

• Challenge: General-purpose STT models (e.g., Whisper) and aviation-specific models trained on controlled airspace perform poorly on non-towered radio calls due to noise, abbreviations, and flexible identification formats (e.g., "Skyhawk" vs. full tail number).
• Solution: The system uses a Large Language Model (LLM) pipeline with domain-specific context:
  • Static Context: Runway numbers, airport names, and standard terminology.
  • Dynamic Context: A real-time list of aircraft in the vicinity (ADS-B IDs, models, distance, and direction) retrieved from the FAA N-Number database.
  • Process: Audio is transcribed using gpt-4o-transcribe, and the LLM (Gemma 3 27B) uses the context to identify the speaker and extract the intent.

B. Intent Extraction

• Radio calls are converted into a discrete set of intent labels ($I$) to create a compact representation for the prediction model.
• Label Categories:
  • Traffic Pattern Legs: Entering/leaving specific segments (Upwind, Crosswind, Downwind, Base, Final) for specific runways.
  • Operations: Takeoff, Landing.
  • Departure Directions: Cardinal directions (N, E, S, W) for leaving the pattern.
  • Fallbacks: "Unknown" (no call in 10 mins), "Insufficient Info," and "Other" (ground ops).
• Mechanism: The extracted label $\ell$ is mapped to a dense vector embedding ($h_{int}$) via a learnable embedding layer.

C. Goal Prediction Model (Multimodal Architecture)

The core prediction engine is a probabilistic model that outputs a distribution over future goal locations.

• Inputs:
  1. Observed Trajectory ($x(t)$): Processed by a Temporal Convolutional Network (TCN) with causal, dilated 1-D convolutions. The output is pooled via Global Average Pooling (GAP) to create a trajectory vector ($h_{traj}$).
  2. Intent Embedding ($h_{int}$): The vector representation of the radio call intent.
• Fusion: $h_{traj}$ and $h_{int}$ are concatenated and passed through a shared Multi-Layer Perceptron (MLP).
• Output: Three parallel heads predict the parameters of a Gaussian Mixture Model (GMM) with $K=15$ components:
  1. Mean Head: Predicts the center ($\mu_k$) of each Gaussian.
  2. Variance Head: Predicts log-variances ($\log \sigma_k^2$).
  3. Weight Head: Predicts mixture coefficients ($\pi_k$).
• Training: The model is trained end-to-end using Negative Log-Likelihood (NLL) loss plus an entropy loss to encourage mode diversity.

3. Key Contributions

1. Robust ASR for Non-Towered Airspace: A method using LLMs with dynamic/static context to achieve high accuracy in transcribing and identifying speakers in unstructured general aviation radio calls.
2. Intent-to-Trajectory Mapping: A novel framework to extract discrete intent labels from natural language and fuse them with trajectory data for goal prediction.
3. Empirical Validation: Experimental proof that language conditioning significantly reduces goal prediction error on real-world data from the TartanAviation dataset (Pittsburgh-Butler Regional Airport).

4. Experimental Results

Experiments were conducted on the TartanAviation dataset (KBTP airport), comparing the proposed method against baselines like TrajAirNet, GooDFlight, and constant velocity models.

• Speech Recognition Performance:

  • Including domain context reduced the Word Error Rate (WER) from 60.55% to 33.97%.
  • Speaker Identification Accuracy (SIA) improved from 63.6% to 94.8%.
  • Intent Labeling Accuracy (ILA) improved from 32.8% to 82.8%.
  • Insight: Even with moderate WER, LLMs successfully extracted semantic meaning.

• Goal Prediction Accuracy (Final Displacement Error - FDE):

  • The proposed method achieved a Mean FDE of 0.486 km on the 7daysJune subset.
  • This outperformed the best trajectory-only baseline (TrajAirNet rerun: 1.390 km) by a significant margin.
  • The method showed a statistically significant improvement over all baselines except GooDFlight (which uses a much larger diffusion model but lacks language conditioning).
  • Ablation Study: Removing language conditioning (Leave-One-Feature-Out) increased FDE by +0.598 km, confirming that intent labels provide substantial predictive value beyond motion history.

• Sensitivity Analysis:

  • Observation Horizon: Increasing the observation window did not significantly improve accuracy, suggesting current position and intent are more critical than long-term history.
  • Prediction Horizon: Language conditioning maintained better accuracy than trajectory-only models as the prediction horizon extended (120s).
  • Call Recency: The model remained robust even when conditioning on radio calls up to 10 minutes old, indicating that intent persists over time.

5. Significance and Conclusion

• Safety Impact: The improved prediction accuracy (reducing error to ~0.48 km) brings autonomous aircraft closer to meeting safety standards requiring a minimum separation of 1,500 feet (0.457 km) in terminal airspace.
• Paradigm Shift: This work demonstrates that socially aware robotics in aviation requires interpreting unstructured human communication, not just sensor data.
• Future Directions: The authors note limitations, including the need to integrate this into a closed-loop control system, handling multi-aircraft interactions simultaneously, and developing systems for autonomous aircraft to generate their own radio calls.

In summary, the paper establishes that language conditioning is a critical enabler for safe autonomous flight in non-towered airspace, bridging the gap between human pilot communication and robotic motion planning.