Language Model Security Database

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Commercial Multimodal Large Language Model (MLLM) integrated systems are vulnerable to a "Dual Steganography" jailbreak paradigm (referred to as Odysseus). The vulnerability arises from the reliance of safety filters on the assumption that malicious content must be explicitly visible in the input or output modalities (text or image). Attackers can bypass these filters by encoding malicious queries into binary matrices and embedding them into benign-looking images using steganographic encoders…

Large Language Models (LLMs) that use special tokens to define conversational structure (e.g., via chat templates) are vulnerable to a jailbreak attack named MetaBreak. An attacker can inject these special tokens, or regular tokens with high semantic similarity in the embedding space, into a user prompt. This manipulation allows the attacker to bypass the model's internal safety alignment and external content moderation systems. The attack leverages four primitives: 1. Response Injection…

Large Language Model (LLM) inference-time watermarking schemes are vulnerable to evasion via character-level perturbations that disrupt the model's tokenizer. Unlike token-level attacks (e.g., synonym replacement), character-level edits—such as homoglyph substitutions, zero-width character insertions, and typos—force the tokenizer to segment a single semantic unit into multiple sub-word tokens. This fragmentation alters the context window used by the watermarking hashing function (e.g., the…

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Large Language Models (LLMs) equipped with native code interpreters are vulnerable to Denial of Service (DoS) via resource exhaustion. An attacker can craft a single prompt that causes the interpreter to execute code that depletes CPU, memory, or disk resources. The vulnerability is particularly pronounced when a resource-intensive task is framed within a plausibly benign or socially-engineered context ("indirect prompts"), which significantly lowers the model's likelihood of refusal compared…

Large Language Models (LLMs) with structured output APIs (e.g., using JSON Schema) are vulnerable to Constrained Decoding Attacks (CDAs). CDAs exploit the control plane of the LLM's decoding process by embedding malicious intent within the schema-level grammar rules, bypassing safety mechanisms that primarily focus on input prompts. The attack manipulates the allowed output space, forcing the LLM to generate harmful content despite a benign input prompt. One instance of a CDA is the Chain Enum…