🤖 machine learning

Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels

This contribution examines the trade-off between compression and latency in real-time text transmission via LLM-based entropy coding over fixed-rate channels and shows that while Huffman coding is optimal for oversized channels with zero algorithmic latency, larger language models significantly reduce bitrates, thereby altering the selection of the optimal coder, with results validated across a 25-fold parameter range from GPT-2 to Llama 3.2.

Original authors: Vishnu Teja Kunde, Jean-Francois Chamberland, Krishna R. Narayanan, Jamison Ebert

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Vishnu Teja Kunde, Jean-Francois Chamberland, Krishna R. Narayanan, Jamison Ebert

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a live radio broadcast, in which someone is reading a book aloud, to a friend. The friend's radio has a fixed speed limit for how fast it can receive information (like a narrow pipe). The reader speaks at a steady pace, but the chosen words are unpredictable. Some words are very common (like "the"), while others are rare and surprising (like "wonderful").

This article is about the best method for packing these words into a digital data stream so they arrive quickly and clearly without getting stuck in a traffic jam.

The Core Idea: Prediction is Compression

The authors use a clever trick: predicting the future.
Imagine a super-smart assistant (an AI) standing next to the reader. Before the reader says a word, the assistant guesses what it will be.

If the assistant is 99% sure the reader will say "the," it only needs to send a tiny, almost invisible signal to confirm this.
If the reader says something surprising, the assistant sends a longer, more detailed signal.

The better the assistant is at guessing, the fewer "bits" (digital building blocks) are needed to send the message. This is the "predict-and-encode" method.

The Traffic Jam Problem

Here is the catch: The reader speaks at a steady pace, but the "size" of the digital signal for each word changes.

Common words = small signals.
Rare words = large signals.

These signals are sent through a pipe with a fixed speed. If some large signals arrive simultaneously, they queue up in a waiting line (a queue) before they can be sent. This causes a delay. The reader may have finished a sentence, but the listener hears it only after a few seconds because the "large signals" are stuck in traffic.

The article asks: How do we balance keeping the signals small (compression) while ensuring they do not get stuck in traffic (delay)?

The Candidates: Different Packing Methods

The researchers tested five different methods for packing these signals:

The Perfect Theorist (Shannon): This is the "magical" method. It uses exactly the right amount of space for each word, no more and no less. It has no delay, but it is impossible to build in real life because it requires sending "fractional" bits (like 0.5 of a building block).
The Instant Packer (Huffman): This method uses only whole building blocks. It is slightly less efficient than the perfect theorist (it wastes a tiny bit of space), but it has a huge advantage: It can be unpacked immediately. As soon as a word arrives, the listener knows exactly what it is. No waiting.
The Patient Packer (Arithmetic Coding): This is almost as efficient as the "perfect theorist." However, it is like a puzzle where you only see the picture once you have all the pieces. The listener must wait and collect a series of bits from future words before they can decode the current word. This creates an inherent delay, no matter how fast the computer is.
The Group Packer (rANS): This method waits to pack words in batches (like waiting for a bus to fill up before it departs). If you wait for a large bus (a large batch), you save a lot of space. But the first person in line must wait until everyone else has arrived before the bus departs. This creates a delay that depends on the size of the group.
The Old-Fashioned Packer (Gzip): This is a standard method for files, but it is terrible for live speech. It tries to find patterns in raw text without using the smart AI assistant. In the end, huge signals are sent, causing massive traffic jams.

The Big Discovery: Larger AI Changes the Rules

The researchers tested this with two AI assistants:

GPT-2: A smaller, older assistant.
Llama 3.2: A much larger, smarter assistant (25 times larger).

The Result:
The smarter assistant (Llama) was so good at guessing the next word that the signals became incredibly small. Because the signals were so small, they no longer piled up in traffic. The pipe was suddenly "oversized" (it had much extra space).

What This Means for Choosing a Method:

When the AI is weak: The signals are large. The traffic jam is real. You need the "patient packer" (Arithmetic Coding) to squeeze every bit of space out of the pipe, even if this means the listener has to wait a few seconds.
When the AI is strong: The signals are tiny. The traffic jam disappears. In this case, the "instant packer" (Huffman) is the winner. It is fast, simple, and the tiny bit of wasted space does not matter because the pipe has so much free room.

The Analogy of the "Human Factor"

The article also notes that people in real life do not speak at a perfect, uniform speed. Sometimes they speak quickly, sometimes slowly. To ensure a radio system works for the fastest speaker, engineers must build the pipe wide enough to handle the fastest talker.

This means that for the average person, the pipe is actually huge and empty. Since the pipe is so wide, the "instant packer" (Huffman) is almost always the best choice. You do not need complex, delayed methods because traffic jams rarely occur.

The Conclusion

The article concludes that the problem shifts as AI intelligence advances. We need to worry less about selecting a complex encoding algorithm to save space. Instead, we should focus on making the AI predictor better.

If the AI is smart enough, the simplest and fastest method (Huffman) becomes the best choice because the "traffic" is so light that we do not need to pack the cars tightly together. The trade-off is no longer about "how to pack the box," but rather "how smart the person is who guesses what goes into the box."

Technical Conclusion: Real-Time Text Transmission via LLM-Based Entropy Coding over Fixed-Rate Channels

Problem Statement
This work investigates the inherent trade-off between compression and latency in real-time text transmission systems employing "predict-then-code" strategies. In such systems, a causal Large Language Model (LLM) predicts the next token in a sequence, and an entropy coder converts these probabilities into a variable-length bitstream. This bitstream is subsequently transmitted over a fixed-rate channel.

The central challenge arises from the interaction between the variable bit lengths of tokens (determined by text predictability) and the constant service rates of the channel. This discrepancy creates a queue where the latency per token depends on the mean and variance of bit lengths, the algorithmic latency of the coder, and the channel capacity. While previous studies have focused on compression rates, the latency implications of various coding schemes under real-time conditions—particularly the distinction between computational latency (hardware-dependent) and algorithmic latency (inherent to the coder's structure)—remain largely unexplored.

Methodology
The authors develop a queuing model for streaming text transmission, where characters arrive at a fixed rate ( $\lambda$ ), are tokenized, predicted by an LLM, and encoded into bits. These bits enter a First-In-First-Out (FIFO) queue served by a channel with constant capacity $C$ .

The study evaluates six coding schemes across two LLM scales (GPT-2 with 124M parameters and Llama 3.2 with 3B parameters) using five public domain texts:

Shannon Coding: An idealized benchmark assigning $-\log_2 q(x_n|x_{<n})$ bits per token. It exhibits no algorithmic latency but is practically unrealizable.
Huffman Coding: An adaptive, context-dependent code assigning integer, prefix-free codewords. It offers no algorithmic latency but incurs overhead due to integer bits.
Arithmetic Coding (AC): A streaming scheme approaching the Shannon limit with high precision but introducing delayed decodability. The decoder must accumulate bits from future tokens before resolving the current one, generating inherent algorithmic latency.
rANS (Asymmetric Numeral Systems): Evaluated with various block sizes ( $K$ ). It buffers $K$ tokens before encoding, trading compression efficiency (via amortization of state overhead) for increased buffer latency.
Gzip (DEFLATE): Used with per-token flush ( $Z\_SYNC\_FLUSH$ ) to ensure immediate decodability. It ignores LLM predictions, relying instead on its own dictionary, leading to significantly higher bit costs.

The experiments simulate a source rate of $\lambda = 20$ characters per second (approx. 240 words per minute). The channel rate $C$ is varied relative to the model-based source rate to analyze performance in over-provisioned, capacity-near, and under-capacity regimes.

Main Contributions

Queuing Model: Development of a model separating algorithmic latency (inherent to the coder) from computational latency, analyzing how variable codewords interact with fixed-rate channels in real time.
Empirical Scaling Analysis: Evaluation of the compression-latency trade-off across a 25-fold parameter range (GPT-2 vs. Llama 3.2), demonstrating how improved model quality alters the optimal coding strategy.
Identification of the Pareto Frontier: Mapping the compression-latency boundary to identify operating regimes where specific coders (e.g., Huffman vs. Arithmetic) are preferred.

Results

Impact of Model Quality: Llama 3.2 reduced cross-entropy compared to GPT-2 by approximately 38% (measured in bits per character). This reduction effectively "over-provisions" the channel for a fixed capacity, shifting the system from a capacity-near regime to an over-provisioned one.
Coder Performance:
- Arithmetic Coding: Achieved near-optimal compression (0% overhead) but suffered from significant algorithmic latency (approx. 2.5 seconds of lookahead at tested rates) due to delayed decodability.
- Huffman Coding: Achieved Shannon-near compression with no algorithmic latency. Under GPT-2, it showed a slight negative overhead (−1%) relative to Shannon due to integer rounding effects; under Llama 3.2, the overhead was modest (~5%).
- rANS: Demonstrated a clear trade-off where larger block sizes ( $K$ ) improved compression but increased latency linearly. However, modern tokenizers (BPE/SentencePiece) already group characters into units of 3–4 characters, meaning the 1-bit overhead of Huffman is sufficiently amortized, rendering the compression advantage of rANS less critical in many regimes.
- Gzip: Performed poorly (10–12x Shannon overhead) due to the inability to leverage LLM predictive power and the costs per token flush.
Regime Shifts: In over-provisioned channels (typical when sizing for the fastest speakers in a real system), Huffman coding proved the practical choice due to its simplicity and lack of algorithmic latency. The compression advantage of arithmetic coding was critical only near the capacity limit.

Significance and Claims
The work argues that the landscape of real-time coding has fundamentally shifted due to modern tokenizers and powerful LLMs.

Tokenization as a Solution: By grouping characters into tokens, modern tokenizers reduce the relative impact of Huffman's 1-bit integer overhead, making Huffman coding a viable, latency-free alternative to more complex schemes like rANS or Arithmetic Coding.
Shift in Design Trade-offs: The primary design challenge is no longer selecting the most efficient coder (an algorithmic question) but choosing the appropriate predictor (an AI engineering question). Improving the predictor (e.g., switching from GPT-2 to Llama 3.2) reduces the source rate, potentially shifting the system into an over-provisioned regime where simpler, latency-free coders become optimal.
Practical Recommendation: For real-time speech/text transmission, where channels are typically sized for peak rates (over-provisioning the average case), Huffman coding combined with an LLM predictor is the pragmatic choice. It offers no algorithmic latency, simple implementation, and a compression overhead negligible relative to the conditional entropy of the tokens.

The authors conclude that while Arithmetic Coding remains theoretically near-optimal, its inherent latency is often unjustified in practical, over-provisioned scenarios. The focus should shift to optimizing the predictor's quality and inference costs rather than the coder's algorithmic complexity.