Language Model Security Database

Large Language Model (LLM) inference APIs that expose top-k logits or log-probabilities are vulnerable to model extraction and cloning. An attacker can execute a two-stage attack to replicate the proprietary model without access to weights, gradients, or training data. First, by submitting fewer than 10,000 random queries and aggregating the returned unrounded logits, the attacker recovers the model's output projection matrix using Singular Value Decomposition (SVD). Second, the attacker…

A vulnerability in LLM-based agents, dubbed AI Agent Injection (AI²), allows attackers to hijack the agent's actions by manipulating the agent's memory retrieval mechanism. The attack involves two main steps: (1) Stealing action-aware knowledge from the agent's memory using crafted adversarial queries targeting the retriever module and (2) Generating Trojan prompts consisting of a Trojan string and hijacking instructions. The Trojan string is designed to manipulate the retriever into…