Language Model Security Database

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

A logical vulnerability exists in LLM security proxies and guardrails that utilize weighted-average aggregation algorithms for multi-turn risk scoring. The scoring logic exhibits a mathematical "ceiling property" where the cumulative conversation-level risk score converges to the per-turn score regardless of the number of interaction turns ($n$). Consequently, the aggregated score is bounded by the maximum single-turn score ($cum \leq \max(s_i)$). This allows remote attackers to bypass…

A vulnerability exists in Large Language Model (LLM) and Large Reasoning Model (LRM) serving interfaces that allow user-defined response prefixes, such as plain text-completion (v1/completions), Fill-in-the-Middle (FIM), or assistant message prefilling. An attacker can perform a Response Prefix Attack (RPA) by injecting maliciously crafted Chain-of-Thought (CoT) reasoning tokens immediately following the assistant's start delimiter (e.g., <|im_start|>assistant). Because these tokens are placed…

Commercial Multimodal Large Language Model (MLLM) integrated systems are vulnerable to a "Dual Steganography" jailbreak paradigm (referred to as Odysseus). The vulnerability arises from the reliance of safety filters on the assumption that malicious content must be explicitly visible in the input or output modalities (text or image). Attackers can bypass these filters by encoding malicious queries into binary matrices and embedding them into benign-looking images using steganographic encoders…

Reasoning-specialized Large Language Models (LLMs) that utilize Chain-of-Thought (CoT) processes are vulnerable to reasoning-exploitation jailbreaks. Attackers can bypass standard safety alignments (such as RLHF) by using adaptive multi-turn interactions or semantic transformations to induce the model to generate intermediate reasoning steps that "rationalize" or "contextualize" a harmful request. Because current alignment techniques often fail to scale linearly with reasoning depth, forcing…

Large Language Models (LLMs) that use special tokens to define conversational structure (e.g., via chat templates) are vulnerable to a jailbreak attack named MetaBreak. An attacker can inject these special tokens, or regular tokens with high semantic similarity in the embedding space, into a user prompt. This manipulation allows the attacker to bypass the model's internal safety alignment and external content moderation systems. The attack leverages four primitives: 1. Response Injection…

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

A vulnerability termed "Trojan Horse Prompting" exists in conversational multimodal models, specifically demonstrated on Google’s Gemini-2.0-flash-preview-image-generation. The vulnerability allows an attacker to bypass safety alignment mechanisms (RLHF and SFT) by manipulating the structural protocol of the conversational API. Unlike standard jailbreaks that manipulate the user prompt, this attack exploits "Asymmetric Safety Alignment" by forging a conversational history where the role is…

Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…