cs.IT papers | Gist.Science

The intersection of computer science and information theory explores how we quantify, transmit, and process data efficiently. This field lies at the heart of modern digital communication, tackling fundamental questions about the limits of compression, the reliability of signals over noisy channels, and the mathematical structures that underpin our connected world. It is the invisible engine driving everything from secure encryption to the massive data streams powering the internet.

At Gist.Science, we bridge the gap between these complex concepts and broader understanding by monitoring every new preprint in this category as it appears on arXiv. Our team processes each paper to provide both accessible plain-language overviews and detailed technical summaries, ensuring you can grasp the core innovations without getting lost in dense notation.

Below are the latest papers in this rapidly evolving domain, curated to help you stay informed about the newest breakthroughs in computer science and information theory.

💬 NLP

Information-Theoretic Limits of Reliability and Scaling in Language Models

This paper challenges the assumption that perfect reliability is achievable through scaling alone by establishing an information-theoretic framework that defines inherent reliability ceilings based on task ambiguity and inter-token dependencies, thereby deriving a unified scaling law that identifies the bottleneck between training data and model capacity while explaining phenomena like retrieval augmentation and catastrophic forgetting.

Subhabrata Majumdar2026-07-17

🔢 mathematics

Capability from Access Structure, Not Scale: Lower Bounds and Pre-Registered Tests for Hybrid Sequence Models

This paper challenges the notion that scaling alone guarantees capability convergence by proposing the Capability Convergence Hypothesis, which argues that true capability requires a specific hybrid architecture combining a compressive state channel and a verbatim-index channel, a claim supported by theoretical lower bounds and pre-registered experimental results.

Wenhui Chen, Jianlin Chen, Ziyao Lin, Chi Man Vong2026-07-17

🔢 mathematics

Exact Online Rank Recycling in Floyd's Uniform Subset Sampler

This paper demonstrates that Floyd's subset sampler admits an exact round-local factorization of its internal ordering coordinate, enabling the precise recycling of this randomness into a residual state to achieve a complete $k!$ state-space factorization without binomial arithmetic, while proving that such immediate rank recycling is invalid for partial Fisher-Yates arrays.

Yingqi Zhang (Department of Computer Science,Technology, Tsinghua University, Beijing, China)2026-07-17

🔢 mathematics

Capacity of Uniform Noise Channels Under Average Input Power Constraints

This paper resolves the long-standing open problem of determining the capacity of additive uniform noise channels under average input power constraints by precisely characterizing the capacity and the corresponding optimal input and output distributions through a novel periodization identity and Fourier analytic techniques.

Yihan Zhang2026-07-17

🔢 mathematics

Better Privacy Guarantees for Larger Groups

This paper establishes that for private histograms with fixed disjoint groups, the optimal privacy budget dependence on group size $n$ is an inverse-square rate of $O(n^{-2})$ , which is both achievable via a shifted-log Gaussian mechanism and necessary for any mechanism satisfying count-dependent zero-concentrated differential privacy with relaxed error bounds at zero.

JacK Fitzsimons2026-07-17

🔢 mathematics

Decision Making Needs Uncertainty Quantification [Lecture Notes]

This lecture note establishes that optimal decision-making under uncertainty requires matching the representation of uncertainty (such as posterior distributions, prediction sets, or credal sets) to both the agent's risk profile and their knowledge of the environment, while providing guarantees on the utility actually obtained.

Osvaldo Simeone2026-07-17

📊 statistics

Precise sample covariance spectral norm error -- an RDT view

This paper employs a novel Random Duality Theory (RDT) framework, combining explicit upper bounds with a new bilinear-quadratic lower-bounding mechanism and a two-replica strategy, to derive the precise limiting value of the spectral norm error for sample covariance matrices of centered Gaussians, thereby moving beyond previous scaling characterizations to provide exact closed-form results.

Mihailo Stojnic2026-07-17

⚡ electrical engineering

Lossy compression of weighted graph adjacency matrices by transform coding

This paper proposes a lossy compression framework for weighted graphs that preserves topology while compressing edge weights by transforming them into signals on a line graph for filter bank processing, quantization, and entropy coding, alongside a novel smoothness measure to predict compression performance without explicitly constructing the line graph.

Kenta Yanagiya, Junya Hara, Hiroshi Higashi, Yuichi Tanaka, Antonio Ortega2026-07-17

🔢 mathematics

Covering Sequences and Covering-Sequences Codes

This paper introduces $(n,R)$ -covering sequences and $(n,m,R)$ -covering-sequences codes as optimal building blocks, demonstrating how Hamming codes can be utilized to construct these structures with short lengths and small cardinalities for both small and large radii.

Tuvi Etzion2026-07-17

🔢 mathematics

A Correlation-Gap Bound for Nonlinear Gaussian PCA

This paper establishes that for nonlinear Gaussian PCA, the standard Karhunen-Loève basis is nearly optimal—within a factor of $1+O(1/\sqrt{d})$ of the best adaptive basis—by proving a correlation-gap bound that demonstrates the advantage of optimizing over all orthonormal bases vanishes as dimension increases.

Minbo Gao, Zhengfeng Ji, Chenghua Liu2026-07-17