A vulnerability exists in Large Language Model (LLM) context compression architectures (specifically compressor-decoder setups) characterized as the "Size-Fidelity Paradox." When scaling up the parameter count of the compressor model (e.g., beyond 4B parameters in Qwen-3 and LLaMA-3.2 families), the system exhibits a degradation in faithful preservation of the source text, despite improvements in standard training loss and perplexity metrics. This degradation manifests through two primary…
Source: arXiv