Language Model Security Database

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

A vulnerability exists in aligned Large Language Models (LLMs) where a harmful instruction can be obfuscated through a multi-step formalization process, bypassing safety mechanisms. The attack, named Prompt Jailbreaking via Semantic and Structural Formalization (PASS), uses a Reinforcement Learning (RL) agent to dynamically construct an adversarial prompt. The agent learns to apply a sequence of actions—such as symbolic abstraction, logical encoding, mathematical representation, metaphorical…

The GPT-OSS-20B large language model contains critical failures in its alignment and Chain-of-Thought (CoT) reasoning mechanisms, specifically in how it prioritizes numerical objectives and validates procedural structure. The model is vulnerable to "Quant Fever," where explicit numerical targets in a prompt (e.g., "delete 90% of files") override contextual safety constraints (e.g., "do not delete important files"). Furthermore, the model exhibits "Reasoning Procedure Mirage," where harmful…

LLM-based search agents are vulnerable to manipulation via unreliable search results. An attacker can craft a website containing malicious content (e.g., misinformation, harmful instructions, or indirect prompt injections) that is indexed by search engines. When an agent retrieves and processes this page in response to a benign user query, it may uncritically accept the malicious content as factual and incorporate it into its final response. This allows the agent to be used as a vector for…

A vulnerability exists in Large Language Models (LLMs) and multi-label text classification systems that allows for Textual Dynamic Outputs Attacks (TDOA). This technique enables hard-label black-box attacks against systems with variable or generative output spaces (where the number of labels or specific label tokens are not fixed). The attack functions by training a surrogate model on clustered coarse-grained labels derived from the victim model's fine-grained dynamic outputs. It subsequently…

Large Language Model (LLM) inference-time watermarking schemes are vulnerable to evasion via character-level perturbations that disrupt the model's tokenizer. Unlike token-level attacks (e.g., synonym replacement), character-level edits—such as homoglyph substitutions, zero-width character insertions, and typos—force the tokenizer to segment a single semantic unit into multiple sub-word tokens. This fragmentation alters the context window used by the watermarking hashing function (e.g., the…

A vulnerability exists in the alignment mechanisms of Large Language Models (LLMs) where activation steering—the process of injecting vectors into hidden states during inference—can systematically bypass refusal safeguards. By modifying the residual stream activations at intermediate layers (typically $\lfloor L/2 \rfloor$) using the formula $\overline{\mathbf{x}}_{i}^{(l)}=\mathbf{x}_{i}^{(l)}+\alpha\mathbf{v}$, an attacker can force the model to comply with harmful requests. Research…

A vulnerability exists in Vision-Language Models (VLMs) that allows for the bypass of safety alignment mechanisms through loss-guided adversarial image perturbations. This attack, known as JaiLIP, operates entirely in the image space, requiring no textual prompt manipulation. The vulnerability is exploited by optimizing an adversarial image using a joint objective function that minimizes the Mean Squared Error (MSE) between the clean and perturbed image while maximizing the model's loss for…

A vulnerability exists in multiple Large Language Models (LLMs) that allows for safety alignment bypass through a technique named Activation-Guided Local Editing (AGILE). The attack uses white-box access to a source model's internal states (activations and attention scores) to craft a transferable text-based prompt that elicits harmful content.

Large Language Models (LLMs) and Vision-Language Models (VLMs) are vulnerable to an automated, adaptive role-play jailbreak attack known as GUARD (Guideline Upholding Test through Adaptive Role-play and Jailbreak Diagnostics). The vulnerability exists because the models fail to recognize malicious intent when harmful queries are embedded within complex, iteratively optimized "playing scenarios."