AVERY: Intent-Driven Adaptive VLM Split Computing via Embodied Self-Awareness for Efficient Disaster Response Systems
AVERY is an intent-driven adaptive split computing framework that enables efficient Vision-Language Model deployment on resource-constrained UAVs for disaster response by dynamically partitioning computation into functional Context and Insight streams based on operator intent and network conditions, thereby achieving high accuracy and low energy consumption in unstable environments.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a rescue coordinator in a chaotic disaster zone, like a flooded city or a wildfire area. You have a fleet of drones (UAVs) flying overhead, but they are small, battery-powered, and flying through stormy weather with spotty internet connections.
You need these drones to do two very different jobs:
- The "Quick Glance": "Hey, is the whole neighborhood underwater? Are there any people on the roofs?" You need this answer right now, even if it's a bit blurry.
- The "Deep Dive": "Zoom in on that red roof. Is that a person trapped under a car, or just a pile of debris?" You need a super-clear, detailed answer, but you can wait a few seconds longer for it.
The Problem:
Current drones are like smart but simple cameras. They can see "water" or "fire," but they can't understand complex questions like "Find the person trapped under the car." To do that, they need a super-smart AI brain (called a Vision-Language Model or VLM).
But here's the catch:
- The "Super Brain" is too heavy: It's like trying to carry a supercomputer in your backpack. It drains the drone's battery in minutes.
- The "Internet" is broken: Sending a full, high-definition video stream to a cloud server to get an answer is impossible because the disaster zone has no signal or very slow internet.
The Solution: AVERY
The researchers built a system called AVERY (think of it as a smart, adaptive drone pilot). Instead of forcing the drone to do everything itself or sending everything to the cloud, AVERY splits the work intelligently based on what you actually need to know (your "intent").
Here is how AVERY works, using a simple analogy:
1. The Two-Stream Brain (The "Eyes" and the "Detective")
AVERY splits the AI's brain into two distinct pathways, inspired by how human eyes work:
- The "Context Stream" (The Quick Glance):
- Analogy: Imagine a security guard who quickly scans a room and shouts, "I see a fire!" or "I see a person!"
- How it works: This uses a lightweight, fast AI. It sends tiny, compressed data packets. It's perfect for answering broad questions like "Is the area safe?" It runs constantly, keeping you updated in real-time without draining the battery.
- The "Insight Stream" (The Deep Dive):
- Analogy: Imagine a forensic detective who needs to examine a specific clue under a microscope.
- How it works: This uses the heavy, super-smart AI. It sends detailed data to the cloud to answer complex questions like "Is that a human or a dog?" It only turns on when you specifically ask for it.
2. The "Smart Switch" (The Embodied Self-Awareness)
This is the magic part. AVERY has a tiny onboard controller that acts like a traffic cop. It constantly checks two things:
- What do you want? (Your Intent: "Just tell me if there's a fire" vs. "Show me exactly who is trapped.")
- What is the weather? (The Network: Is the internet fast and stable, or slow and broken?)
How the Switch Works:
- Scenario A: You ask, "Is anyone on the roof?"
- The Switch: "Okay, that's a 'Quick Glance' question. I'll use the Context Stream. I'll send a tiny, fast update. No need to wait for the cloud."
- Scenario B: You ask, "Highlight the person on the roof."
- The Switch: "Okay, that's a 'Deep Dive' question. I need the Insight Stream."
- The Twist: If the internet is terrible right now, the Switch doesn't just give up. It says, "I can't send the full high-definition file, so I'll compress it just enough to get through, or I'll send a slightly lower-quality version so you get an answer now rather than waiting forever."
3. The "Compression Magic"
When the drone needs to send the "Deep Dive" data, it doesn't send the whole raw video. It uses a special "bottleneck" (like squeezing a toothpaste tube) to shrink the data down to the smallest size possible while keeping the important details. It's like sending a high-quality photo via text message instead of a 4K video file.
Why This Matters
- Battery Life: By not running the heavy AI all the time, the drone can fly 16 times longer than if it tried to do everything itself.
- Reliability: It works even when the internet is terrible. It adapts on the fly.
- Smarter Decisions: It gives you the right kind of answer for the right moment. You don't get a slow, detailed report when you need a fast warning, and you don't get a vague warning when you need to save a life.
In Summary:
AVERY is like a rescue drone that knows when to be a fast scout and when to be a careful detective. It listens to your questions, checks the internet connection, and automatically decides the smartest way to get you the answer without running out of battery or getting stuck in a signal dead zone. It turns a dumb camera into a truly intelligent, adaptive rescue partner.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.