Language Model Security Database

API gateways can pool distinct customers into one upstream cache identity. The authors observe cross-customer cache reuse in five deliberately pooled gateway configurations through two provider interfaces.

Automatic Prefix Caching (APC) in multi-tenant LLM serving systems introduces a timing side-channel vulnerability that permits cross-tenant data leakage. APC shares computed Key-Value (KV) tensors across different users when their requests share identical initial tokens. Because reusing cached tensors is significantly faster than recomputing them, a measurable difference in Time-To-First-Token (TTFT) exists between cache hits and misses. An attacker can exploit this shared cache by sending…

A vulnerability in multi-tenant LLM serving frameworks allows attackers to reconstruct the private prompts of other users via an active Key-Value (KV) cache side-channel. Frameworks that utilize shared KV caches alongside specific scheduling policies, such as Longest Prefix Match (LPM), prioritize waiting requests based on the length of their matched prefix tokens. An attacker can exploit this by iteratively sending batches of guessed tokens mixed with dummy queries. If a guessed token matches…

LLM serving frameworks utilizing continuous batching and PagedAttention (such as vLLM, SGLang, and Orca) are vulnerable to a resource exhaustion Denial-of-Service attack known as "Fill and Squeeze." An unprivileged remote attacker can exploit the deterministic state transitions of the scheduler's memory management to induce severe latency or service denial. The attack leverages a side-channel vulnerability where Inter-Token Latency (ITL) correlates linearly with global KV-cache usage due to…

Multi-tenant Large Language Model (LLM) inference systems utilizing global Key-Value (KV) cache sharing are vulnerable to a timing side-channel attack. By measuring the Time-To-First-Token (TTFT) latency of crafted API requests, an unprivileged remote attacker can determine if specific token sequences have been previously processed and cached by the system for other users. This observable timing difference between cache hits (low TTFT) and cache misses (high TTFT) allows for the token-by-token…