A vulnerability in multi-tenant LLM serving frameworks allows attackers to reconstruct the private prompts of other users via an active Key-Value (KV) cache side-channel. Frameworks that utilize shared KV caches alongside specific scheduling policies, such as Longest Prefix Match (LPM), prioritize waiting requests based on the length of their matched prefix tokens. An attacker can exploit this by iteratively sending batches of guessed tokens mixed with dummy queries. If a guessed token matches…
Source: arXiv