Zero-Shot Knowledge Base Resizing for Rate-Adaptive Digital Semantic Communication

Technical Summary: Zero-Shot Knowledge Base Resizing for Rate-Adaptive Digital Semantic Communication

Problem Statement
Digital semantic communication systems, particularly those built on the Vector Quantized Variational Autoencoder (VQ-VAE) framework, rely on a shared discrete semantic Knowledge Base (KB) or codebook to transmit information. In these systems, the transmission rate is directly governed by the size of the KB ($K$). A larger $K$ allows for finer-grained semantic representation and higher reconstruction quality but increases the transmission payload (requiring $\lceil \log_2 K \rceil$ bits per index) and computational overhead. Conversely, a smaller $K$ reduces bandwidth and latency but degrades quality.

The fundamental limitation of current VQ-VAE-based systems is that the KB size $K$ is a fixed hyperparameter determined prior to training. To adapt the transmission rate to dynamic channel conditions, existing approaches require training and storing a separate, complete model for each desired KB size. This practice is computationally and storage-prohibitive, preventing truly granular, on-the-fly rate adaptation. While some methods attempt to approximate flexibility by selecting from discrete pre-defined configurations or using variable sub-KBs, they fail to support arbitrary resizing or do not reduce the actual transmission rate effectively.

Methodology
The authors propose a zero-shot KB resizing method that enables the dynamic adaptation of a KB to any smaller target size without retraining. The core premise is that the semantic relationships within a well-trained KB are inherently hierarchical. The method operates on a single, large "parent" KB (size $\hat{K}$) and extracts a smaller, high-quality KB (size $K < \hat{K}$) through a three-step framework:

1. Hyperbolic Embedding for Semantic Granularity:
   The vectors of the pre-trained parent KB are embedded from their native Euclidean space into a hyperbolic space (specifically the Poincaré ball model). This geometric transformation leverages the property that hyperbolic space naturally represents hierarchical structures with low distortion. In this mapping, vectors representing coarse-grained, foundational concepts are positioned near the origin, while vectors representing fine-grained, specific concepts are pushed toward the boundary. The distance from the origin thus serves as a proxy for semantic importance.

2. Hierarchical Pruning via Minimum Spanning Tree (MST):
   Using the hyperbolic distances between vectors, the authors construct a "master semantic tree" using a Minimum Spanning Tree algorithm (Prim's algorithm). The vector closest to the origin (the most general concept) is designated as the root. This tree explicitly defines parent-child relationships among the KB vectors. To resize the KB to a target size $K$, the method performs deterministic pruning: it iteratively removes the leaf nodes (representing the most specific concepts) until exactly $K$ nodes remain. This process preserves the foundational "skeleton" of the knowledge base while discarding the most granular details.

3. Euclidean Projection:
   The pruned set of vectors, now residing in hyperbolic space, is mapped back to the original Euclidean space using the logarithmic map (the inverse of the exponential map used in step 1). The resulting set of vectors forms the final, resized KB, which is fully compatible with existing digital semantic communication systems that operate in Euclidean space.

Key Contributions

• A Principled Three-Step Framework: The paper introduces a complete method for zero-shot KB resizing that utilizes hyperbolic embeddings to reveal semantic hierarchy, MST construction to formalize this hierarchy, and iterative leaf pruning to achieve instant resizing.
• Elimination of Retraining Overhead: The method allows for the creation of KBs of arbitrary sizes from a single trained parent model, eliminating the need to train separate models for different rate levels.
• Robustness at Low Rates: The approach demonstrates superior stability at very low transmission rates compared to models trained from scratch for small KB sizes.

Experimental Results
The authors evaluated the method using an end-to-end image semantic transmission task on the ImageNet dataset, comparing their "Zero-shot resizing" approach against a baseline of "Trained from scratch" models (where a dedicated model is trained for each specific KB size).

• Reconstruction Quality: Across a wide range of KB sizes, the Structural Similarity Index Measure (SSIM) of the zero-shot resized KBs was nearly identical to that of the dedicated baseline models, with an average difference of only 0.8%.
• Robustness: At very low rates (e.g., $K=256$), the baseline models trained from scratch suffered from catastrophic failure, producing images with severe artifacts and unnatural colors. In contrast, the zero-shot method maintained semantic coherence, producing recognizable (albeit blurry) images. This indicates that the hierarchical pruning preserves the most essential concepts, ensuring graceful degradation rather than system collapse.
• Efficiency: The proposed method reduced the total training time and GPU resource requirements by approximately 9-fold. Instead of training nine separate models for nine different KB sizes, only a single parent model was trained; all other sizes were generated via the computationally inexpensive pruning process.

Significance
The paper claims to resolve a fundamental architectural limitation of VQ-VAE-based digital semantic communication systems. By enabling the instant restructuring of a KB to an arbitrary size without retraining, the method offers a practical and efficient path toward flexible, rate-adaptive semantic communication. This capability is critical for future wireless networks where channel conditions and resource constraints are dynamic, allowing systems to balance reconstruction quality and transmission overhead on-the-fly without the prohibitive costs associated with maintaining multiple specialized models.