Technical Summary: Large Language Models Do Not Always Need Readable Language

1. Problem Statement

Large Language Models (LLMs) currently operate under a unified paradigm where knowledge, instructions, and outputs are all encoded in human-readable natural language. While this design optimizes for human controllability and readability, it introduces significant inefficiencies for machine-to-machine communication. Human language contains substantial redundancy (syntax, discourse markers, narrative coherence) that aids human comprehension but reduces semantic density.

In long-context and agentic systems, this redundancy creates a persistent bottleneck regarding context overhead. Existing context compression methods (e.g., token selection, abstractive summarization) generally operate within the constraints of human-readable natural language. Furthermore, learned internal representations often require access to hidden states or additional training, limiting their utility in black-box API settings. The central question addressed by this paper is: If the receiver is an LLM rather than a human, must semantic information still be encoded in fully human-readable natural language?

2. Methodology: BabelTele

The authors introduce BabelTele, a class of model-centric textual representations that explicitly relax human readability as a default constraint. Rather than a fixed protocol, BabelTele is treated as an empirical probe into an LLM's capacity to generate and interpret compact, non-standard textual forms.

Core Principles

BabelTele is elicited through zero-shot prompting based on three principles:

1. Omnilingual Lexical Selection: Relaxing single-language constraints to select high-density lexical units across languages and scripts.
2. Symbolic Collapse: Replacing verbose linguistic structures with compact symbols, emojis, mathematical/logical operators, and punctuation.
3. Recoverable Semantic Density: Preserving semantic details sufficient for capable LLMs to interpret the output without an external codebook.

The methodology treats BabelTele not as a competitive compression algorithm but as a family of high-density representations. The authors evaluate this across multiple dimensions: compression ratio, semantic fidelity, human readability, cross-model transferability, and downstream utility.

3. Key Results

3.1 Decoupling Readability from Decodability

Experiments demonstrate that BabelTele representations are significantly less readable to humans (Dale-Chall score of 16.70, 80.19% difficult-word ratio) and have much lower likelihood under base language models (order-of-magnitude higher Perplexity) compared to original text or natural-language summaries.

• Human vs. Model Performance: Human readers show a significant drop in QA accuracy on BabelTele inputs. Conversely, instruction-tuned LLMs (e.g., Gemini 3.1 Pro) maintain high accuracy, indicating that the representations preserve core semantic structures despite sacrificing natural-language typicality.
• Information Density: BabelTele maintains 99.5% semantic fidelity even when text volume is condensed to 27.9% of its original length.

3.2 Efficiency and Cognitive Overhead

• Accuracy-Retention Frontier: BabelTele forms a more favorable accuracy-retention frontier compared to standard abstractive summaries and extractive baselines (LLMLingua-2). It maintains higher downstream QA accuracy under strong compression, particularly on the MeetingBank dataset.
• Chain-of-Thought (CoT) Overhead: While stronger compression generally increases the number of response CoT tokens (as models reconstruct evidence), BabelTele does not introduce unique overhead. Its token multiplier is often comparable to or lower than summaries, suggesting that the cost is intrinsic to sparse evidence reasoning rather than specific to the BabelTele format.

3.3 Cross-Model Transferability

A critical finding is that BabelTele exhibits zero-shot cross-model transferability.

• Compressed inputs generated by one model (e.g., Gemini 3.1 Pro) can be reliably interpreted by heterogeneous models (e.g., Qwen, GPT-5.4, Kimi) without fine-tuning.
• However, portability is not absolute; it depends on the compressor-reader pair. Aggressive compressors may produce symbolic forms that are harder for other models to decode, and performance retention varies by model family.
• Comprehension does not scale monotonically with model size; larger models do not necessarily perform better on BabelTele inputs than smaller ones, suggesting the phenomenon relies on specific robustness to symbolic forms rather than just scale.

3.4 Downstream Task Performance

• Multi-Agent Communication: In both homogeneous and heterogeneous agent settings, BabelTele reduced inter-agent communication overhead by 38.96% to 44.21% while maintaining competitive task scores (96.6%–99.7% of baseline).
• Agent Memory: On the LoCoMo benchmark, BabelTele incurred minimal accuracy loss compared to original text and preserved more actionable details than standard summarization.
• Context Window Extension: When inputs exceed the context window, BabelTele-based chunk compression outperformed direct truncation (e.g., 62.07% vs. 55.17% accuracy on Code Repo QA), allowing models to capture broader evidence within limited windows.

4. Significance and Claims

The paper claims that human readability, natural-language typicality, and model-side semantic recoverability can be partially decoupled.

• Model-Native Representations: The findings suggest that intermediate representations in LLM systems can be optimized for model decodability and semantic density rather than human fluency.
• General Capability: The ability to interpret these representations appears to be a general capability of instruction-tuned LLMs across diverse architectures, rather than an artifact of a specific model or training setup.
• Practical Utility: BabelTele offers a path toward reducing context overhead in long-context and multi-agent workflows without sacrificing downstream performance, provided the compressor-reader pair is compatible.

The authors position BabelTele not as a finished protocol but as an empirical demonstration that "model-native" textual communication is a viable and efficient alternative to human-readable natural language for machine-to-machine interaction.

5. Limitations and Risks

The authors note that the evaluation is limited to specific benchmarks and model families, and the underlying mechanisms of how LLMs form these representations remain unexplained. They also highlight potential risks: transforming text into non-standard representations could alter behavior in safety-critical domains, and the benchmarks used may contain inherent biases.