Attack Type

Jailbreak Vulnerabilities

Methods for bypassing model safety measures

703 matching entries

Jailbreak research results

703 entries

Browser Agent Jailbreak

Updated 12/29/2024

Refusal-trained Large Language Models (LLMs) show decreased safety when deployed as browser agents compared to their performance in chatbot settings. Attack methods effective at jailbreaking LLMs in chat contexts also successfully bypass safety mechanisms in browser agents, leading to the execution of harmful behaviors. This vulnerability stems from a lack of generalization of safety training to agentic, real-world interaction scenarios and the increased context available to the agent (browser…

Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents

Affects: o1-preview, o1-mini, GPT-4 Turbo +5 more

Source: arXiv

End-of-Sentence MLP Jailbreak

Updated 12/29/2024

Instruction-tuned Large Language Models (LLMs) are vulnerable to jailbreaks via the manipulation of Multi-Layer Perceptron (MLP) neuron weights in end-of-sentence inferences. By selectively re-weighting these neuron activations, an attacker can bypass the model's safety mechanisms and elicit harmful responses. The vulnerability is independent of the specific prompt and generalizable across various models, impacting both prompt-specific and prompt-general attacks.

Jailbreak Instruction-Tuned LLMs via end-of-sentence MLP Re-weighting

Affects: Gemma 2 27B, Gemma 2 2B, Llama 3 70B +4 more

Source: arXiv

Enhanced Jailbreak Transferability

Updated 12/29/2024

A novel jailbreak attack, dubbed SI-GCG, against Large Language Models (LLMs) leverages a fixed harmful template and optimized suffix selection to bypass safety mechanisms and elicit harmful responses with high transferability. The attack utilizes a scenario induction template and a refined optimization process to improve the consistency and effectiveness of the jailbreak across different LLMs. The vulnerability stems from the inability of current safety measures to adequately defend against…

Boosting jailbreak transferability for large language models

Source: arXiv

Ensemble Black-box Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Transferable Ensemble Black-box Jailbreak Attacks on Large Language Models

Affects: Deepseek-v2.5, Gemma 2B IT, Gemma 2 9B IT +5 more

Source: arXiv

Faster GCG LLM Jailbreak

Updated 12/29/2024

Faster-GCG is an optimized jailbreak attack that exploits vulnerabilities in aligned Large Language Models (LLMs) by efficiently finding adversarial prompt suffixes. The attack leverages gradient information to iteratively refine a harmful prompt, overcoming limitations of prior methods like GCG by incorporating a regularization term to improve gradient approximation, using deterministic greedy sampling, and preventing self-looping during optimization. This allows for significantly higher…

Faster-GCG: Efficient discrete optimization jailbreak attacks against aligned large language models

Affects: GPT-3.5 Turbo, GPT-4 Turbo, Llama 2 7B Chat +1 more

Source: arXiv

Fine-tuning Safety Bypass

Updated 12/29/2024

Large Language Models (LLMs), even those initially aligned for safety, are vulnerable to having their safety mechanisms compromised through fine-tuning on a small number of adversarially-crafted or even seemingly benign sentences. Fine-tuning with as few as 10 toxic sentences can significantly increase the model's compliance with harmful instructions.

Locking down the finetuned llms safety

Affects: GPT-3.5 Turbo, GPT-4, Llama 3 70B Chat +4 more

Source: arXiv

Fuzzy Backdoor Jailbreak

Updated 12/29/2024

AdvBDGen demonstrates a novel backdoor attack against LLMs aligned using Reinforcement Learning with Human Feedback (RLHF). The attack generates prompt-specific, fuzzy backdoor triggers, enhancing stealth and resistance to removal compared to traditional constant triggers. The attacker manipulates prompts and preference labels in a subset of RLHF training data to install these triggers. The triggers are designed to evade detection by a "weak" discriminator LLM while being detectable by a…

AdvBDGen: Adversarially Fortified Prompt-Specific Fuzzy Backdoor Generator Against LLM Alignment

Affects: BERT, Gemma 7B, GPT-4 +4 more

Source: arXiv

Gibberish-Suffix LLM Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to jailbreaking via the addition of adversarial suffixes generated by models like AmpleGCG-Plus. These suffixes, often consisting of gibberish or nonsensical text, cause the LLM to bypass safety protocols and generate harmful or undesired outputs. The vulnerability stems from the LLM's inability to reliably identify and filter these adversarial suffixes, even when they lack semantic meaning. AmpleGCG-Plus significantly improves the success rate and…

AmpleGCG-Plus: A Strong Generative Model of Adversarial Suffixes to Jailbreak LLMs with Higher Success Rates in Fewer Attempts

Affects: GPT-3.5 Turbo, GPT-4, GPT-4o +6 more

Source: arXiv

Gradient Image Jailbreaks

Updated 12/29/2024

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…

Gradient-based jailbreak images for multimodal fusion models

Affects: Chameleon 30B, Chameleon 7B, LLaVA 1.6 Llama 3

Source: arXiv

Homotopy-Based LLM Jailbreak

Updated 12/29/2024

Large Language Models (LLMs) are vulnerable to jailbreak attacks utilizing a novel Functional Homotopy (FH) optimization method. FH exploits the functional duality between model training and input generation, iteratively solving a series of "easy-to-hard" optimization problems to generate adversarial prompts that circumvent safety mechanisms and elicit undesirable model responses. This is achieved by first misaligning the model via gradient descent on continuous parameters, then leveraging…

Functional Homotopy: Smoothing Discrete Optimization via Continuous Parameters for LLM Jailbreak Attacks

Affects: Mistral 7B v0.3

Source: arXiv

Related research categories