Cognition Envelopes for Bounded Decision Making in Autonomous UAS Operations

This paper introduces "Cognition Envelopes" as a framework to constrain errors in AI-driven autonomous Uncrewed Aerial Systems by establishing reasoning boundaries, demonstrating their application through a probabilistic clue analysis pipeline for Search and Rescue missions while outlining key software engineering challenges for their implementation.

The Gist

Imagine you have a brilliant, super-smart assistant who can see the world through a camera, read maps, and make complex plans instantly. This assistant is powered by AI (specifically Large Language Models and Vision-Language Models). You send this assistant on a mission to find a lost hiker in a vast forest using a drone.

The AI is great at spotting things. It sees a broken pair of glasses on a rock and thinks, "Aha! The lost hiker was here! I should immediately fly the drone to the other side of the mountain to look for them!"

But here's the problem: The AI is sometimes confident but wrong. It might "hallucinate" (make things up), misread the situation, or suggest a plan that is physically impossible or a waste of time. If you let the AI fly the drone blindly, it might crash, get lost, or waste the battery chasing a ghost.

This paper introduces a solution called a Cognition Envelope.

The Analogy: The "Smart Bouncer" vs. The "Drunk Genius"

Think of the AI as a Drunk Genius.

• The Genius: It has incredible ideas and can solve problems fast.
• The Drunk: It sometimes stumbles, sees things that aren't there, and suggests crazy plans (like "fly the drone into the sun").

Now, imagine you have a Smart Bouncer standing at the door of the club (the decision-making process). This bouncer is the Cognition Envelope.

• The Safety Envelope (The Old Guard): This is like a physical fence around the club. It stops the drone from flying too high or hitting a tree. It handles physical safety.
• The Cognition Envelope (The New Guard): This is the bouncer who checks the ideas. Before the AI's plan gets executed, the Bouncer asks:
  • "Does this plan make sense with what we actually know?"
  • "Is the lost person likely to be there based on how long they've been gone?"
  • "Do we have enough battery to fly there?"
  • "Are you just making this up because you're 'hallucinating'?"

If the AI's plan passes the Bouncer's check, the drone goes. If the plan is crazy or risky, the Bouncer stops it and says, "Hold on, human operator, you need to check this."

How It Works in the Paper (The "Lost Hiker" Story)

The researchers tested this with a real-world scenario: Search and Rescue (SAR) using small drones (sUAS).

1. The Clue: The drone spots a clue (like a backpack or broken glasses).
2. The AI's Job: The AI looks at the picture, guesses what it is, and decides where to search next.
3. The Cognition Envelope's Job:
   • The "Probability Check" (pSAR): The envelope runs a math model. It asks: "If the person was last seen here 2 hours ago, and they are an elderly man, is it physically possible for them to be at the location the AI suggested?" If the AI suggests searching a mountain peak that would take 10 hours to reach, the envelope says, "Nope, that's impossible."
   • The "Cost Check" (MCE): The envelope checks the battery. If the AI wants to fly 50 miles but the drone only has 10 miles of battery left, the envelope says, "Nope, you'll crash."

The Results: Does It Work?

The researchers ran hundreds of simulated missions.

• Without the Envelope: The AI sometimes suggested wild, impossible plans.
• With the Envelope: The "Bouncer" caught the bad ideas.
  • When the AI suggested a search area that made sense, the envelope said "Go!" (Autonomy).
  • When the AI suggested something risky or impossible, the envelope said "Stop, ask a human" (Safety).

They found that the AI was actually pretty good at spotting the clues, but it needed the Envelope to check if the plan to search those clues was logical.

Why This Matters

We are moving toward a future where AI runs our cars, planes, and medical devices. We can't just trust the AI to be perfect because it makes mistakes.

• Old Way: We build a fence (Safety Envelope) to stop the car from driving off a cliff.
• New Way (This Paper): We also need a Logic Check (Cognition Envelope) to stop the car from driving into a wall because the AI "thought" the wall was a door.

The "Open Challenges" (The Homework)

The authors admit this isn't perfect yet. They list some things they still need to figure out:

• Who watches the Watchman? How do we make sure the Cognition Envelope itself isn't making mistakes?
• When to call a human? If the AI asks for help too often, humans get annoyed. If it asks too late, it's dangerous. Finding the right balance is hard.
• Explaining the "No": If the envelope blocks a plan, it needs to tell the human why in simple terms, not just "Error 404."

Summary

Cognition Envelopes are like a reality check for AI. They don't stop the AI from thinking; they just make sure the AI's thoughts are grounded in reality, math, and common sense before they turn into action. It's the difference between a genius who is also a bit crazy, and a genius who is supervised by a sensible manager.

Technical Summary

1. Problem Statement

Autonomous Cyber-Physical Systems (CPS), particularly small Uncrewed Aerial Systems (sUAS) used in Search and Rescue (SAR), are increasingly relying on Foundation Models (Large Language Models - LLMs, and Vision-Language Models - VLMs) for perception, inference, and planning. While these models enhance autonomy, they introduce critical failure modes such as hallucinations, over-generalizations, and context misalignments.

• The Gap: Existing safety mechanisms (Safety Envelopes) constrain physical states (e.g., geofencing, altitude limits) but do not validate the semantic correctness or evidential grounding of the decisions made by AI.
• The Risk: An AI might generate a plan that is physically safe (the drone won't crash) but operationally flawed (e.g., searching an area with zero probability of finding the victim based on current evidence), leading to mission failure or resource waste.
• The Need: A mechanism is required to bound AI reasoning by validating decisions against external evidence, uncertainty, and resource constraints, distinct from the model's internal self-critique (metacognition).

2. Methodology: Cognition Envelopes

The authors introduce Cognition Envelopes, a runtime assurance layer designed to gate the execution of foundation-model pipelines.

2.1 Formal Definition

A Cognition Envelope is defined as a tuple $E = \langle d, e, M, s, G \rangle$:

• $d$ (Decision): The candidate decision/action proposed by the foundation model.
• $e$ (Evidence): External evidence and runtime context available at the time of decision.
• $M$ (Semantic Model): An external, independent model that evaluates $d$ under $e$ to produce a semantic acceptability signal $s$.
• $s$ (Signal): A score, constraint satisfaction outcome, or set of triggered checks.
• $G$ (Gating Rule): A rule that uses $s$ to determine whether to Accept, Alert (human review), or Reject the decision.

2.2 Application Domain: SAR Clue Analysis Pipeline (CAP)

The concept is tested in a SAR scenario where sUAS detect visual clues (e.g., a backpack, broken glasses) and generate search plans.

• The Pipeline (CAP): A 4-stage LLM/VLM pipeline:
  1. Captioner: Describes the clue.
  2. Relevance Checker: Assesses if the clue relates to the lost person (using RAG).
  3. Task Planner: Proposes search areas (terrain subclusters) based on the clue.
  4. Triager: Decides execution strategy (drone pool vs. human review).
• The Cognition Envelope Implementation:
  The envelope acts as an external validator ($M$) consisting of two components:
  1. Probability-Based SAR Model (pSAR): A semantic model using Probability of Area (POA). It calculates the likelihood of a lost person being in a specific area based on:
     • Reachability: Physical accessibility from the Last Known Point (LKP) considering terrain, time, and physiology.
     • Affinity: Behavioral attraction to features like trails, water, or roads.
     • Dynamic Updates: The model updates probabilities when a clue is detected.
  2. Mission Cost Evaluator (MCE): A heuristic check on resource consumption (time, battery).

2.3 Gating Logic

The envelope evaluates the CAP's proposed search area against the pSAR's probability distribution using:

• Percentile Rank ($r$): How the proposed area ranks against all possible areas.
• Ratio-to-Top ($\rho$): The magnitude of the proposed area's probability relative to the best area.
• Entropy ($H_{norm}$): Measures model uncertainty.
• Adaptive Thresholds: The acceptance criteria ($r_{accept}, \rho_{accept}$) tighten when uncertainty is low and loosen when uncertainty is high, balancing autonomy with safety.

3. Key Contributions

1. Conceptual Framework: Definition of "Cognition Envelopes" as a distinct layer from Safety Envelopes and Metacognition, specifically targeting semantic validity and evidential grounding.
2. System Architecture: A working implementation integrating a multi-modal LLM pipeline (CAP) with a probabilistic semantic validator (pSAR) and a cost evaluator (MCE).
3. Experimental Validation: A rigorous evaluation framework using 10 SAR mission vignettes (based on real historical data) and 350 unique test cases involving varied clues, environments, and elapsed times.
4. Software Engineering Insights: Identification of open challenges in designing, validating, and assuring Cognition Envelopes, including ground-truth alignment, verifying the verifier, and human-in-the-loop integration.

4. Results

The system was evaluated across 350 tests to answer two research questions:

• RQ1: Where are external checks needed?

  • The CAP's internal meta-reasoning (Stages 1 & 2) was found to be largely sufficient for clue interpretation and relevance assessment (95% accuracy).
  • Conclusion: External Cognition Envelopes are most critical at Stages 3 & 4 (Task Planning and Triaging), where decisions involve probabilistic reasoning about reachability and resource allocation.

• RQ2: How effective are the safeguards?

  • Without Clue Updates: When pSAR did not update probabilities based on the new clue, it correctly rejected 44% of plans that were outside the expected search area (Group 2 tests), preventing wasted resources.
  • With Clue Updates: When pSAR updated its model to incorporate the clue's location, the approval rate for relevant search plans increased significantly. This demonstrates that the envelope allows for higher autonomy when evidence supports the decision, while still blocking high-risk, low-probability searches.
  • Key Inhibitors: Distance from LKP, Elapsed Time, and terrain barriers (rivers/cliffs) were the primary factors triggering rejections or alerts.

5. Significance

• Trustworthy Autonomy: The paper provides a practical pathway to deploy LLMs in life-critical systems by decoupling "creative" planning from "assured" validation.
• Beyond Safety: It shifts the focus from physical safety (will the drone crash?) to operational validity (is the plan logical given the evidence?).
• Generalizability: The authors demonstrate that the Cognition Envelope concept is applicable beyond SAR, citing potential use cases in UAV exploration, multi-robot formation control, and precision oncology (validating treatment plans against medical guidelines).
• Software Engineering Roadmap: The paper highlights that integrating these systems requires new SE practices, specifically regarding the specification of envelope roles, handling uncertainty in evidence, and ensuring the "verifier" itself is verified.

In summary, the paper argues that for AI-supported CPS to be truly autonomous and safe, they must operate within Cognition Envelopes that enforce reasoning boundaries based on external evidence and probabilistic reality, rather than relying solely on the model's internal confidence.