⚡ electrical engineering

Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

This paper proposes a collaborative intelligence paradigm where multiple large language models distributed across devices and the cloud work together through vertical and horizontal collaboration to deliver high-quality responses under diverse resource constraints, while also addressing training strategies and open research challenges.

Original authors: Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a team of workers trying to solve a complex puzzle. Some workers are small, fast, and work right on your desk (your phone or laptop), but they aren't very smart. Others are giants with massive libraries of knowledge, but they work in a distant office (the cloud), and it takes time to send them a message and get an answer back. Plus, calling the giants costs money, and sometimes the phone line is broken.

This paper proposes a new way for these workers to team up. Instead of trying to make the small worker smarter or forcing the giant to come to your desk, they simply talk to each other to solve the problem together. The authors call this "Collaborative Intelligence."

Here is a breakdown of their ideas using simple analogies:

1. The Problem: Why One Worker Isn't Enough

The Small Worker (On-Device): Think of a smartphone. It's fast and private, but it's like a student who has read a few books. It can handle simple questions, but if you ask it to write a complex legal contract or analyze a video, it might get confused or run out of energy.
The Giant (The Cloud): Think of a super-intelligent expert in a data center. They know everything but are far away. Sending a question to them takes time (latency), costs money per word, and requires a strong internet connection. If the internet cuts out, you're stuck.
The Dilemma: You can't just use the small worker for everything (bad answers) or the giant for everything (too slow, too expensive, or impossible if the internet is down).

2. The Solution: A Team Huddle

The paper suggests that these workers should collaborate at the task level. They don't need to share their "brains" (the internal code or weights of the AI); they just need to share messages (questions, answers, and notes).

The authors describe two ways this team can huddle:

Vertical Collaboration (The Assistant and the Boss):
- Imagine a local assistant (your phone) who tries to answer a question first. If they think, "I'm not sure, this is too hard," they quickly ask the Boss (the cloud) for help.
- The Trick: The assistant doesn't just dump the whole conversation history to the Boss every time (which is slow and expensive). Instead, they summarize the key points before asking, saving time and money.
Horizontal Collaboration (The Peer Group):
- Imagine a group of friends (like several robots or phones) working on a project. They don't have a boss; they just talk to each other.
- Debate Style: They all guess an answer, then argue about who is right until they agree.
- Division of Labor: One person looks up facts, another does the math, and a third writes the report.
- The Challenge: If they talk too much, they waste time and data. The paper suggests they need to learn how to talk efficiently so they don't just repeat themselves.

3. Learning to Work Together

The paper argues that we shouldn't just program these workers with rigid rules (like "If the question is hard, call the cloud"). Instead, we should teach them to learn how to collaborate.

Learning to Route: The small worker learns to judge its own confidence. It learns, "I can answer this easily, so I'll do it myself to save money," or "This is tricky, and I have budget left, so I'll call the cloud." It learns to manage a "budget" of time and money over a whole conversation, not just for one question.
Learning to Cooperate: The workers are trained to stop acting like solo artists. Instead of just waiting for instructions, they learn to adapt to each other. For example, if one worker is slow, the others learn to wait or summarize their thoughts more clearly so the group finishes faster.

4. A Real-World Example: The Kitchen Assistant

The paper tests this idea with a "smart kitchen" scenario.

The Setup: A user is cooking and talking to their phone. The phone has a camera (seeing the food) and a small AI brain. The cloud has a super-smart AI.
The Goal: Answer questions about the cooking process while staying under a strict time limit (30 seconds total) and a strict money limit ($0.05 total).
The Result:
- If the phone tries to do everything alone, the answers are okay but not great.
- If the phone sends everything to the cloud, the answers are perfect, but it runs out of money and time.
- The Winner: The "Learned Router." This system learns to send simple questions to the phone and hard questions to the cloud, but it also decides exactly which photos to send to the cloud to save data. It manages the budget like a smart shopper, stretching the $0.05 to get the best possible answer.

5. The Remaining Hurdles

The paper admits this isn't perfect yet. Two big challenges remain:

Scaling Up: What happens if you have 1,000 robots or cars trying to talk to each other? The network might get clogged with too much chatter. We need better ways to organize these large swarms.
Trust and Safety: If a bad actor tricks one worker in the chain, that bad idea could spread to everyone else, like a rumor. Also, if the workers share private photos or medical data to solve a problem, how do we make sure that data doesn't leak?

Summary

In short, this paper says: Don't try to make every device a super-computer. Instead, let small devices and big cloud computers talk to each other like a team. By teaching them to decide when to help each other and how to communicate efficiently, we can get the best of both worlds: fast, private, and smart answers without breaking the bank or the internet.

Technical Summary: Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

1. Problem Statement

The deployment of Large Language Models (LLMs) faces a fundamental dichotomy: frontier models with hundreds of billions of parameters offer superior capabilities but require resource-rich cloud environments, while lightweight models suitable for on-device execution suffer from significant capability gaps on complex tasks. Existing solutions, such as model partitioning, speculative decoding, and federated learning, operate at the model level. These approaches require white-box access to model internals, tightly coupled execution, and often assume homogeneous hardware.

Consequently, they fail to address scenarios involving:

Intermittent connectivity (e.g., UAVs, field robots).
Strict latency budgets (e.g., closed-loop control).
Data-residency constraints (e.g., healthcare, finance).
Sustained high-volume inference bounded by API rate limits and per-token costs.

No single endpoint (device or cloud) can deliver high-quality service across this entire spectrum of resource constraints (computation, memory, communication, and cost). The paper argues for a shift from model-level distribution to Collaborative Intelligence, a paradigm where multiple independent, black-box LLMs distributed across network tiers collaborate at the task level via natural language or structured messages, without sharing weights or internal states.

2. Methodology and System Design

The paper proposes a framework for collaborative inference organized along two complementary and composable dimensions: Vertical Device-Cloud Collaboration and Horizontal Multi-Agent Collaboration.

A. Vertical Device-Cloud Collaboration

This topology addresses the capability gap between lightweight on-device LLMs and powerful cloud LLMs.

Joint LLM and Modality Selection: The system must decide which endpoint serves a request and, for multi-modal inputs, which subset of data (e.g., specific camera views) to transmit. Transmitting all inputs is often unnecessary; optimizing input composition reduces communication costs while preserving quality.
Context Management: In multi-turn conversations, switching endpoints requires transferring accumulated dialogue history. The paper highlights the tension between frequent switching (high context transfer overhead) and session commitment (loss of adaptability). Practical solutions involve context caching and selective summarization to compress state before transmission.

B. Horizontal Multi-Agent Collaboration

This topology enables peer LLM agents to solve tasks collectively.

Collaboration Patterns: The paper identifies three primary patterns:
1. Debate: Agents generate independent responses, critique, and revise until consensus.
2. Division-of-Labor: Tasks are decomposed into subtasks (e.g., retrieval, computation, synthesis) assigned to specialized agents.
3. Hierarchical: A supervisor agent decomposes tasks and monitors progress, while worker agents execute subtasks.
Communication Topology: The choice of topology (fully connected, star, relay) critically impacts overhead. Since messages are natural language rather than compact vectors, fully connected debate scales poorly ( $O(N^2T^2)$ token traffic). Hierarchical structures are noted as more communication-efficient.

C. Learning to Collaborate

The paper distinguishes between learning to route (vertical) and learning to cooperate (horizontal).

Routing Policy Learning:
- Router-Based Selection: A lightweight classifier predicts the best endpoint based on query features, trained via constrained optimization to maximize quality under cost/latency budgets.
- Self-Routing via Post-Training: The on-device LLM is trained via reinforcement learning (RL) to attempt tasks first and decide whether to escalate based on internal reasoning signals (e.g., self-consistency, entropy). This eliminates the need for an external router and leverages the model's own capability assessment.
Cooperative Capability Learning:
- Cooperative Policy Optimization: Moving beyond static prompt engineering, agents are co-trained via RL in authentic collaborative scenarios. The paper notes that training agents in isolation is insufficient; simultaneous co-training allows agents to adapt to evolving strategies.
- Inter-Agent Network Optimization: Training incorporates communication costs directly, encouraging agents to use sparse topologies and concise, high-information messages to reduce token traffic while maintaining performance.

3. Key Results and Case Study

The paper presents a case study on Device-Cloud Routing in Multi-Modal Conversations (a kitchen assistant scenario) to validate the proposed learning approach.

Setup: A lightweight text-only LLM (Phi-3-mini) on a Jetson TX2 device collaborates with a cloud multi-modal frontier LLM (GPT-4o). The system operates under strict cumulative budgets (30s latency, $0.05 cost).
Findings:
- On-device only: Fast and free but yields low response quality.
- Cloud only: High quality but frequently violates budget constraints.
- Prompt-based Routers (LLM-as-Router): Trace a standard quality-latency-cost tradeoff curve; higher quality necessitates proportionally higher latency and cost.
- RL-based Learned Router: Breaks the standard tradeoff curve. By internalizing the remaining budget as part of the policy's state, the learned router reasons about resource consumption over the entire conversation rather than per-query. It achieves comparable response quality to cloud-only approaches at substantially lower latency and cost.

4. Open Challenges and Research Directions

The paper identifies two primary axes of open challenges:

Scaling Under Resource Heterogeneity:
- Developing protocols that adapt to constantly shifting ecosystems where endpoints vary by orders of magnitude in capability and cost, without requiring retraining.
- Scaling from pairwise collaboration to large-scale swarms (e.g., drone fleets), addressing scheduling, consensus over volatile links, and long-context memory pressure.
Trustworthy Collaborative Intelligence:
- Privacy: Balancing the need for context sharing (for quality) with the containment of sensitive data (PII, medical records) across trust boundaries.
- Security: Defending against cascading prompt injection attacks where a compromise at one node propagates through the collaboration hierarchy, potentially causing real-world harm in physical systems.

5. Significance and Claims

The paper claims that Collaborative Intelligence represents a distinct and necessary paradigm for networked LLMs. Its significance lies in:

Task-Level Orchestration: Unlike prior model-level approaches, it orchestrates cooperation among heterogeneous, black-box endpoints (including proprietary APIs) via semantic exchanges.
Breaking Tradeoffs: Through learned routing policies that treat resource budgets as stateful, temporal constraints, the system can achieve quality-latency-cost balances unattainable by static or per-query decision-making.
Scalability: It provides a framework for combining vertical offloading and horizontal multi-agent collaboration into hybrid topologies, enabling complex workflows (e.g., cloud-coordinated UAV fleets) that no single endpoint could handle alone.

The authors conclude that realizing this vision requires advances in adaptive protocols for heterogeneous ecosystems and robust safety mechanisms for distributed, autonomous agents.