Language Model Security Database

A "Helpful Mode" role-confusion vulnerability exists in specific Large Language Model (LLM) safety guardrails, specifically Nemotron-Safety-8B and Granite-Guardian-3.2-5B. These models, designed to act as binary classifiers (outputting "Safe" or "Unsafe") for content moderation, can be manipulated via contextually framed adversarial prompts (e.g., academic research requests, corporate security scenarios, or roleplay) to abandon their classification objective. Instead of blocking the request…

Large Vision Language Models (LVLMs) are vulnerable to a jailbreaking attack that combines image typography manipulation with multi-turn prompting. The vulnerability exploits the model's visual encoder and instruction-following capabilities by embedding a harmful textual query directly into a benign image as a visible caption (using specific fonts and blending techniques). An attacker then engages the model in a three-turn conversation: first asking a benign question about the visual object…

Embodied Artificial Intelligence (AI) agents utilizing Vision-Language Models (VLMs) for perception and planning are vulnerable to Indirect Environmental Jailbreak (IEJ). The vulnerability arises from the system's failure to distinguish between user-issued instructions and text embedded in the physical environment (e.g., writing on walls, sticky notes, or projections). The VLM processes visual text detected in the camera feed as authoritative context or direct commands, allowing a black-box…

Improper restriction of the "Capability Space" in Large Language Model (LLM) applications allows remote attackers to manipulate application behavior through "Goal Deviation" attacks. This vulnerability arises when developers rely on the broad capabilities of a foundational model (e.g., GPT-4, LLaMA) without implementing sufficient negative constraints or disabling default plugins (e.g., DALL-E, Web Search) in the system prompt. Attackers can exploit this via natural language inputs to trigger…

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

Centralized Multi-Agent Systems (MAS) architectures, specifically those implemented using frameworks such as Magentic-One, LangGraph, and OpenAI Agents, contain design-level vulnerabilities facilitating safety alignment bypasses. Two primary mechanisms drive this vulnerability: 1. Context Fragmentation via Atomic Delegation: The orchestrator decomposes adversarial prompts into atomic, granular sub-tasks (e.g., "click button," "write function header") before delegating them to sub-agents. These…

Multiple open-weight Large Language Models (LLMs)—specifically those prioritizing capability over safety alignment—exhibit a critical vulnerability to adaptive multi-turn prompt injection and jailbreak attacks. While these models effectively reject isolated, single-turn adversarial inputs (averaging ~13.11% Attack Success Rate), they fail to maintain safety guardrails and policy enforcement across extended conversational contexts. By leveraging iterative strategies such as "Crescendo" (gradual…

Multimodal Large Language Models (MLLMs) capable of processing speech and audio are vulnerable to Speech-Audio Compositional Attacks. This vulnerability exists because current safety mechanisms often rely on text-only transcription or fail to analyze the full acoustic context of an input. By manipulating the composition of audio signals, an attacker can bypass safety filters and elicit harmful responses. The attacks exploit three specific mechanisms: (1) Speech Overlap, where harmful…

A vulnerability exists in multiple Large Language Models (LLMs) that allows for safety alignment bypass through an advanced jailbreaking technique called Template and Suffix Optimization (TASO). The attack combines two distinct optimization methods in an alternating, iterative feedback loop. First, a semantically meaningless adversarial suffix is optimized (e.g., using gradient-based methods like GCG) to force the LLM to begin its response with an affirmative phrase (e.g., "Sure, here is...")…

Agentic AI browsers and LLM-powered browser extensions are vulnerable to indirect prompt injection via the processing of untrusted web content. The vulnerability arises when the AI agent ingests the Document Object Model (DOM), including hidden elements, HTML comments, metadata, and accessibility labels, into its context window to perform tasks such as page summarization or autonomous navigation. Because the LLM cannot distinguish between system instructions and untrusted external data, an…