Attack Type

Jailbreak Vulnerabilities

Methods for bypassing model safety measures

703 matching entries

Jailbreak research results

703 entries

Adaptive LLM Jailbreaks

Updated 12/28/2024

Leading safety-aligned Large Language Models (LLMs) are vulnerable to simple adaptive jailbreaking attacks. These attacks utilize manually crafted prompt templates, combined with random search on a suffix to maximize the log-probability of a target token indicating compliance (e.g., "Sure"). The attacks are adaptive, as the prompt template and target token are customized for specific models. Furthermore, some models are vulnerable to transfer attacks (using successful prompts from one LLM on…

Jailbreaking leading safety-aligned llms with simple adaptive attacks

Affects: Claude 2.0, Claude 2.1, Claude 3 Haiku +14 more

Source: arXiv

Amplified Adversarial Suffix Generation

Updated 12/29/2024

Large language models (LLMs) are vulnerable to jailbreaking attacks using adversarially generated suffixes. The AmpleGCG attack generates a large number of diverse, effective suffixes which bypass safety mechanisms in both open and closed-source LLMs. The attack leverages the observation that low loss during suffix generation is not a reliable indicator of jailbreaking success, and generates diverse suffixes from intermediate steps of the optimization process.

Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms

Affects: GPT-3.5 Turbo, GPT-4, Llama 2 7B Chat +2 more

Source: arXiv

Fast Adaptive LLM Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to adversarial prompting attacks, where a crafted suffix appended to an instruction causes the LLM to generate unsafe or harmful content. The AdvPrompter technique trains a separate LLM to generate these adversarial suffixes, rapidly bypassing LLM safety mechanisms. The generated suffixes are human-readable and contextually relevant, making them harder to detect than previous methods. The attack is effective against both open-source and closed-source…

Advprompter: Fast adaptive adversarial prompting for llms

Affects: Falcon 7B Instruct, GPT-3.5 Turbo, GPT-4 +6 more

Source: arXiv

LLM Refusal Suppression Jailbreak

Updated 12/28/2024

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that exploit their tendency to refuse harmful requests. The "Don't Say No" (DSN) attack overcomes this refusal mechanism by optimizing prompts to suppress negative responses, increasing the likelihood of generating harmful content. This is achieved by modifying the loss function during adversarial prompt optimization, prioritizing the suppression of refusal keywords over the elicitation of affirmative responses. The attack…

Don't Say No: Jailbreaking LLM by Suppressing Refusal

Affects: GPT-3.5 Turbo, GPT-4

Source: arXiv

Logic-Chain Jailbreak

Updated 1/26/2025

This vulnerability allows attackers to bypass LLM safety mechanisms and elicit malicious content by injecting a chain of benign, semantically equivalent narrations into a seemingly innocuous article. The LLM connects these scattered narrations, effectively executing the malicious intent hidden within the seemingly benign context. This differs from previous attacks which directly embed malicious prompts, making detection by both LLMs and human reviewers more difficult.

Hidden You Malicious Goal Into Benigh Narratives: Jailbreak Large Language Models through Logic Chain Injection

Affects: BERT, GPT, GPT-4 +1 more

Source: arXiv

Multi-Turn Crescendo Jailbreak

Updated 12/28/2024

Large Language Models (LLMs) are vulnerable to the "Crescendo" multi-turn jailbreak attack. This attack uses a series of benign, escalating prompts to gradually lead the LLM into generating harmful or disallowed content, bypassing built-in safety mechanisms. The attack leverages the LLM's tendency to follow conversational patterns and build upon previous responses, making it difficult to detect based solely on individual prompts.

Great, now write an article about that: The crescendo multi-turn llm jailbreak attack

Affects: Claude 2, Claude 3 Opus, Claude 3.5 Sonnet +6 more

Source: arXiv

Vocabulary-Guided LLM Hijacking

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to a vocabulary attack where carefully selected words from the model's vocabulary, identified using an optimization procedure and embeddings from another LLM, are inserted into user prompts. This manipulation can cause the target LLM to generate specific undesired outputs (goal hijacking), such as offensive language or false information, even with minimal word insertions. The attack is difficult to detect because the inserted words may appear…

Vocabulary Attack to Hijack Large Language Model Applications

Affects: Flan-T5 XXL, Llama 2 7B Chat, Llama 2 Chat +1 more

Source: arXiv

Diverse Jailbreak Optimization

Updated 3/4/2025

Large Language Models (LLMs) are vulnerable to jailbreak attacks that utilize an optimized algorithm to bypass safety mechanisms. The vulnerability stems from the redundancy in existing trigger-searching algorithms, resulting in inefficient exploration of the prompt space and allowing attackers to elicit harmful responses. The proposed DPP-based Stochastic Trigger Searching (DSTS) algorithm demonstrates a statistically significant improvement over existing optimization-based attacks.

Enhancing Jailbreak Attacks with Diversity Guidance

Affects: Alpaca 7B, Gemma 7B IT, GPT-3.5 Turbo +10 more

Source: arXiv

LLM Distraction Jailbreak

Updated 12/28/2024

Large Language Models (LLMs) are vulnerable to a novel black-box jailbreak attack, termed "Distraction-based Adversarial Prompts" (DAP). DAP leverages the distractibility and over-confidence of LLMs by concealing malicious queries within complex, unrelated prompts. A memory-reframing mechanism further redirects the LLM's attention away from the distracting context and toward the malicious query, causing the model to bypass safety mechanisms and generate harmful or unintended outputs.

Tastle: Distract large language models for automatic jailbreak attack

Affects: GPT-3.5 Turbo, GPT-3.5-1106), GPT-4 +4 more

Source: arXiv

One-Image VLM Jailbreak

Updated 1/26/2025

A data poisoning attack, termed ImgTrojan, allows adversaries to bypass safety mechanisms in Vision-Language Models (VLMs) by injecting a small number of maliciously crafted image-caption pairs into the training dataset. These poisoned pairs associate seemingly benign images with jailbreak prompts, causing the VLM to generate unsafe outputs when presented with the poisoned images at inference time. The attack's success rate is notably high even with a very low poison ratio (e.g., one poisoned…

ImgTrojan: Jailbreaking Vision-Language Models with ONE Image

Affects: LLaVA 1.5 13B, LLaVA 1.5 7B

Source: arXiv

Related research categories