Attack Type

Jailbreak Vulnerabilities

Methods for bypassing model safety measures

703 matching entries

Jailbreak research results

703 entries

LLM-Tuned Image Jailbreak

Updated 4/21/2025

A vulnerability in safeguarded text-to-image models allows bypassing of safety filters and alignment methods through the use of adversarial prompts generated by a fine-tuned large language model (LLM). The attack, termed PromptTune, effectively rewrites unsafe prompts into semantically similar adversarial prompts that evade safety mechanisms, resulting in the generation of harmful images. The attack does not require repeated queries to the target text-to-image model.

Jailbreaking Safeguarded Text-to-Image Models via Large Language Models

Affects: BLIP-2, CLIP, DALL-E 3 +4 more

Source: arXiv

Latent Subspace Jailbreak

Updated 3/19/2025

A vulnerability exists in large language models (LLMs) where the model's internal representations (activations) in specific latent subspaces can be manipulated to trigger jailbreak responses. By calculating a perturbation vector based on the difference between the mean activations of "safe" and "jailbroken" states, an attacker can introduce a targeted perturbation to the model's activations, causing it to generate unsafe outputs even when presented with a safe prompt. This manipulates the…

Probing Latent Subspaces in LLM for AI Security: Identifying and Manipulating Adversarial States

Affects: Llama 3.1 8B Instruct

Source: arXiv

MLM Adaptive RAG Poisoning

Updated 12/30/2025

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Models

Affects: GPT-4 Turbo, GPT-4o, Claude 3.5 Sonnet +2 more

Source: arXiv

Metaphor-Based LLM Jailbreak

Updated 3/19/2025

Large Language Models (LLMs) are vulnerable to a novel jailbreaking attack leveraging adversarial metaphors. The attack, termed AVATAR, induces the LLM to reason about benign metaphors related to harmful tasks, ultimately leading to the generation of harmful content either directly or through calibration of metaphorical and professional harmful content. The attack exploits the LLM's cognitive mapping process, bypassing standard safety mechanisms.

from Benign import Toxic: Jailbreaking the Language Model via Adversarial Metaphors

Affects: Claude 3.5 Sonnet, GLM 3 6B, GPT-4o +8 more

Source: arXiv

Metaphor-Based T2I Jailbreak

Updated 4/12/2025

A vulnerability in text-to-image (T2I) models allows bypassing safety filters through the use of metaphor-based adversarial prompts. These prompts, crafted using LLMs, indirectly convey sensitive content, exploiting the model's ability to infer meaning from figurative language while circumventing explicit keyword filters and model editing strategies.

Metaphor-based Jailbreaking Attacks on Text-to-Image Models

Affects: DALL-E 3, Flux, Llama 3 8B Instruct +3 more

Source: arXiv

Multimodal Narrative Jailbreak

Updated 4/21/2025

Multimodal Large Language Models (MLLMs) are vulnerable to a novel attack vector leveraging narrative-driven visual storytelling and role immersion to circumvent built-in safety mechanisms. The attack, termed MIRAGE, decomposes harmful queries into environment, character, and activity triplets, generating a sequence of images and text prompts that guide the MLLM through a deceptive narrative, ultimately eliciting harmful responses. The attack successfully exploits the MLLM's cross-modal…

MIRAGE: Multimodal Immersive Reasoning and Guided Exploration for Red-Team Jailbreak Attacks

Affects: Gemini 1.5 Pro, GPT-4V, Grok 2 Vision +3 more

Source: arXiv

Probabilistic Multimodal Jailbreak

Updated 3/19/2025

Multimodal Large Language Models (MLLMs) are vulnerable to Jailbreak-Probability-based Attacks (JPA). JPA leverages a Jailbreak Probability Prediction Network (JPPN) to identify and optimize adversarial perturbations in input images, maximizing the probability of eliciting harmful responses from the MLLM, even with small perturbation bounds and few iterations. The attack operates by modifying the input image's hidden states within the MLLM to increase the predicted jailbreak probability.

Utilizing Jailbreak Probability to Attack and Safeguard Multimodal LLMs

Affects: DeepSeek VL 1.3B, InstructBLIP Vicuna 13B, InternLM XComposer +2 more

Source: arXiv

Schema-Guided LLM Jailbreak

Updated 4/12/2025

Large Language Models (LLMs) with structured output APIs (e.g., using JSON Schema) are vulnerable to Constrained Decoding Attacks (CDAs). CDAs exploit the control plane of the LLM's decoding process by embedding malicious intent within the schema-level grammar rules, bypassing safety mechanisms that primarily focus on input prompts. The attack manipulates the allowed output space, forcing the LLM to generate harmful content despite a benign input prompt. One instance of a CDA is the Chain Enum…

Output Constraints as Attack Surface: Exploiting Structured Generation to Bypass LLM Safety Mechanisms

Affects: Gemini 2.0 Flash, Gemma 2 9B, GPT-4o +5 more

Source: arXiv

Segmented Prompt Jailbreak

Updated 4/3/2025

Large Language Models (LLMs) incorporating safety filters are vulnerable to a "Prompt, Divide, and Conquer" attack. This attack segments a malicious prompt into smaller, seemingly benign parts, processes these segments in parallel across multiple LLMs, and then reassembles the results to generate malicious code, bypassing the safety filters. The attack's success relies on the iterative refinement of initially abstract function descriptions into concrete implementations. Individual LLM safety…

Prompt, Divide, and Conquer: Bypassing Large Language Model Safety Filters via Segmented and Distributed Prompt Processing

Affects: Claude 3.5 Haiku, Claude 3.5 Sonnet, Gemini 1.5 Pro +3 more

Source: arXiv

AP-Test Guardrail Identification

Updated 3/4/2025

This vulnerability allows attackers to identify the presence and location (input or output stage) of specific guardrails implemented in Large Language Models (LLMs) by using carefully crafted adversarial prompts. The attack, termed AP-Test, leverages a tailored loss function to optimize these prompts, maximizing the likelihood of triggering a specific guardrail while minimizing triggering others. Successful identification provides attackers with valuable information to design more effective…

Peering Behind the Shield: Guardrail Identification in Large Language Models

Affects: Aegis Defensive, Aegis Permissive, GPT-4o +8 more

Source: arXiv

Related research categories