Technical Summary: CRONA - Cross-Modal Navigation with Multi-Agent Reinforcement Learning

Problem Formulation

Robust embodied navigation relies on complementary sensory cues (e.g., RGB-D, audio, language). However, obtaining high-quality, well-aligned multi-modal data is difficult in practice. Training monolithic models to process rich multi-modal inputs often leads to complex representations, enlarged policy spaces, and unstable learning due to misaligned modalities (differing in dimensionality, noise, and temporal structure). Furthermore, existing multi-agent collaborative navigation approaches predominantly focus on homogeneous agents (receiving identical modalities) or limited sensory configurations, leaving the potential of heterogeneous collaboration—where agents possess complementary, specialized modalities—largely unexplored.

The authors address this by framing the problem as a fully decentralized collaborative navigation task without inter-agent communication. The setting is modeled as a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), where agents must infer task assignments and learn cooperative policies based on partial local observations (e.g., an audio agent hearing a sound source, a vision agent seeing a visual target) to locate specific objects in complex 3D environments.

Methodology: CRONA Framework

The paper proposes CRONA (Cross-Modal Navigation with Multi-Agent Reinforcement Learning), a decentralized Multi-Agent Reinforcement Learning (MARL) framework designed for cross-modal collaboration. The architecture follows the Centralized Training with Decentralized Execution (CTDE) paradigm.

1. Auxiliary Belief Predictors

To mitigate the stochasticity and noise inherent in raw audio signals, CRONA employs auxiliary belief predictors for agents equipped with audio sensors.

• Function: These predictors estimate control-relevant beliefs from spectrogram observations, specifically predicting the target location (relative to the agent) and target category.
• Mechanism: A convolutional encoder extracts acoustic features. A location head predicts a sound-source goal in global coordinates, transformed into the agent's local frame. A category head predicts a belief distribution over object categories.
• Stabilization: Predictions are smoothed using an exponential moving average to reduce variance. These beliefs are inferred from local histories, ensuring consistency with the information available to decentralized policies during execution.

2. Attention-Based History Encoder

Agents maintain a short-term history cache of observations and actions.

• Encoding: Visual inputs (RGB-D) and audio inputs (spectrograms) are processed by modality-specific encoders (ResNet-18 for vision, CNN for audio).
• Fusion: Encoded features are concatenated with the agent's pose and natural language instructions to form a latent observation embedding.
• Temporal Modeling: A Transformer-based architecture (multi-head attention) processes the sequence of cached observation-action embeddings to capture spatial and temporal dependencies, producing a compact history representation used for policy selection.

3. Centralized Multi-Modal Critic

During training, a centralized critic estimates the joint value function to facilitate learning.

• Inputs: The critic utilizes the joint history embeddings of all agents, the auxiliary beliefs (from audio agents), and the global state (scene layout, target locations, agent poses, task completion status).
• Training Objective: The critic is trained to minimize a clipped surrogate objective against a return target computed via Generalized Advantage Estimation (GAE). The policy update for each decentralized agent is conditioned only on its local history and auxiliary belief, using the shared advantage estimate from the critic.
• Parameter Sharing: Encoders, belief predictors, and history transformers are shared between the decentralized actors and the centralized critic to improve representation learning and accelerate training.

Key Contributions

1. Benchmark Construction: The authors constructed a collaborative navigation benchmark using diverse Matterport3D scenes, featuring tasks where multi-modal agents (vision and/or audio) must cooperate to locate targets.
2. CRONA Framework: They proposed a MARL framework that leverages auxiliary belief predictors and a centralized multi-modal critic to enable effective cross-modal collaboration without inter-agent communication.
3. Modality-Dominance Patterns: Through extensive experiments, the paper identifies and analyzes five distinct patterns of modality dominance across different scenarios:
   • No Clear Dominance: Collaboration outperforms single agents regardless of modality mix.
   • Vision Dominance: Visual cues are superior; audio inputs may introduce noise.
   • Audio Dominance: Acoustic cues are critical (e.g., in corridors); visual inputs may be insufficient.
   • Cross-Modal Dominance: Heterogeneous collaboration (complementary modalities) significantly outperforms homogeneous teams, particularly when targets have clean, modality-specific cues.
   • Multi-Modal Dominance: In highly complex environments, full multi-modal inputs and higher model capacity are required for optimal performance.

Experimental Results

Experiments were conducted on five Matterport3D scenes (Studio, Corridor, Apartment, Ranch, Maze) with varying complexities and target configurations.

• Performance vs. Baselines: CRONA consistently outperformed single-agent baselines and homogeneous collaboration baselines (VLA-Collab, ALA-Collab, AVLA-Collab) in terms of success rate and navigation efficiency in most scenarios.
• Cross-Modal Efficiency: In the "Ranch" scene, where targets had distinct modality-specific cues (camera shutter for pictures, silverware dropping for tables), CRONA achieved a 64.62% success rate, significantly outperforming the homogeneous full-modality baseline (AVLA-Collab at 18.93%). This highlights the efficiency of specialized agents over monolithic models with limited capacity.
• Complexity Handling: In the most complex "Maze" scene, the homogeneous full-modality baseline (AVLA-Collab) achieved the highest success rate (26.16%), suggesting that large, complex environments benefit from richer multi-modal inputs and higher model capacity, though CRONA remained competitive.
• Ablation Studies:
  • Model Capacity: Homogeneous multi-modal agents were highly sensitive to embedding size; CRONA was more stable across different capacities.
  • Signal Quality: CRONA demonstrated greater robustness to low-resolution visual inputs compared to vision-heavy baselines, as audio agents could compensate.
  • Components: The auxiliary location belief and global state information were found to be critical; removing them caused performance to drop by approximately 50%.

Significance and Claims

The paper claims that cross-modal collaboration offers a robust and scalable alternative to monolithic multi-modal navigation. By assigning complementary sensory modalities to specialized agents, CRONA reduces the burden of learning dense, aligned multi-modal representations within a single model while preserving the strengths of each modality.

The authors emphasize that the effectiveness of collaboration is domain-dependent. They argue that:

• Homogeneous collaboration with limited modalities is sufficient for short-range navigation.
• Heterogeneous collaboration is generally efficient and effective when targets possess clean, modality-specific cues.
• Large, complex environments require both richer multi-modal perception and increased model capacity.

The work demonstrates that decentralized agents can coordinate effectively without explicit communication, provided they are trained with a centralized critic that leverages global state and auxiliary beliefs. This approach addresses the challenges of noisy, misaligned, and asynchronous real-world sensory data, offering a practical pathway for robust embodied navigation in diverse environments.