Multilingual Vision-Language Models, A Survey

Technical Summary: Multilingual Vision-Language Models, A Survey

Problem Statement
Vision-language models (VLMs) have achieved unprecedented progress in processing natural language and visual data jointly. However, the field is heavily skewed toward English, with performance significantly lagging for other languages, particularly low-resource ones. A critical, often unaddressed tension exists in developing multilingual VLMs: the conflict between language neutrality (ensuring consistent cross-lingual representations of objective concepts) and cultural awareness (adapting outputs to reflect linguistic, cultural, and contextual nuances). Current training methods can enforce language neutrality to a measurable degree, but cultural awareness relies on diverse data and is difficult to evaluate. Furthermore, there is a discrepancy between training objectives and evaluation goals, as two-thirds of existing benchmarks prioritize semantic consistency through translation-based approaches, often failing to capture culturally grounded content.

Methodology
This survey provides a comprehensive analysis of the state of the art in multilingual VLMs and their evaluation benchmarks. The methodology involves:

1. Model Review: A systematic examination of 33 multilingual vision-language models released between 2020 and 2025. The authors categorize these models by architecture (encoder-only, encoder-decoder, decoder-only, and mixture-of-experts), training data strategies (implicit vs. explicit multilingual support), and parameter scales.
2. Benchmark Analysis: A review of 23 multilingual benchmarks. The authors analyze these datasets based on language coverage, typological diversity (using metrics like Mean Pairwise Syntactic Distance and GramBank Feature Value Inclusion), and evaluation paradigms (translation-based vs. culture-aware).
3. Comparative Assessment: The paper contrasts the architectural evolution from early encoder-based models to modern generative decoder-only architectures and evaluates the extent to which current models and benchmarks address the tension between language neutrality and cultural awareness.

Key Contributions

• Taxonomy of Models and Benchmarks: The paper establishes a detailed taxonomy of 33 open-weight multilingual VLMs and 23 benchmarks, providing statistics on language support, parameter counts, and download/citation metrics.
• Identification of Core Tension: The authors explicitly define and analyze the fundamental trade-off between language neutrality (facilitating cross-lingual transfer via consistent representations) and cultural awareness (requiring context-specific adaptation). They note that while neutrality is mathematically optimizable via contrastive learning on parallel data, cultural awareness lacks explicit training objectives and depends heavily on data composition.
• Evaluation Gap Analysis: The survey highlights a significant gap between the number of languages models claim to support (often 100+) and the number actually evaluated in benchmarks (typically 4–30). It further critiques the dominance of translation-based benchmarks (e.g., XVNLI, xGQA) which prioritize semantic equivalence over cultural diversity.
• Architectural Trends: The paper documents the field's shift from encoder-only models (optimized for classification) to decoder-only and encoder-decoder architectures capable of generative tasks, noting a dramatic increase in parameter counts from hundreds of millions to over 2 trillion.

Results and Observations

• Architectural Evolution: The field has moved from 16 encoder-only models to 14 decoder-only models and 3 encoder-decoder models. Recent decoder-only models (e.g., LLaMA 4, Qwen 3-VL) utilize mixture-of-experts (MoE) architectures and scale to trillions of parameters.
• Language Coverage Disparities: While underlying language encoders (like XLM-RoBERTa) implicitly support 100+ languages, explicit multimodal training data often covers far fewer languages. Many models rely on cross-lingual transfer or synthetic code-switching rather than end-to-end multilingual pre-training.
• Benchmark Limitations: Approximately two-thirds of benchmarks use translation-based methodologies with identical images and translated text, prioritizing language neutrality. Only a minority (e.g., MARVL, ArtElingo-28, CVQA) incorporate region-specific imagery and culturally grounded content.
• Transparency Issues: Commercial open-weight models often lack transparency regarding training data composition and language distribution, making it difficult to assess their true multilingual and cultural capabilities.
• Performance: While models show cross-lingual transfer capabilities, performance on culturally aware tasks remains challenging. Evaluation metrics for generative tasks (e.g., BLEU, chrF) vary across languages and fail to capture cultural appropriateness.

Significance
The paper argues that the field must move beyond the assumption that language neutrality is sufficient for global accessibility. It posits that the tension between neutrality and cultural awareness is a fundamental ethical and technical challenge. The significance of this survey lies in its identification of the current misalignment between training objectives (which favor neutrality) and the need for culturally aware systems. It calls for:

• Development of explicit training objectives for cultural awareness.
• Creation of comprehensive evaluation frameworks that include human-evaluation protocols for cultural appropriateness and cover a broader range of languages and resource tiers.
• Improved data transparency and documentation standards, particularly for commercial models, to enable systematic analysis of data coverage and bias.

The authors conclude that while progress has been made in scaling models and expanding language coverage, the unresolved tension between neutrality and cultural awareness, coupled with evaluation limitations, hinders the deployment of truly inclusive and globally accessible vision-language systems.