Language Model Security Database

A vulnerability in multi-modal large language models (MLLMs) allows attackers to bypass safety mechanisms and elicit harmful responses using a memory-efficient zeroth-order optimization technique. The attack, termed Zer0-Jack, leverages simultaneous perturbation stochastic approximation (SPSA) with patch coordinate descent to generate malicious image inputs, even without access to the model's internal parameters (black-box setting).

Large Language Models (LLMs) are vulnerable to attention-based jailbreak attacks. Attackers can craft prompts that strategically divert the LLM's attention away from sensitive words, causing the model to overlook malicious intent and generate harmful content. This occurs by leveraging the LLM's attention mechanism to focus on benign parts of the prompt while embedding harmful queries within a seemingly harmless context. The success of the attack is correlated with specific attention…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that manipulate attention scores to redirect the model's focus away from safety protocols. The AttnGCG attack method increases the attention score on adversarial suffixes within the input prompt, causing the model to prioritize the malicious content over safety guidelines, leading to the generation of harmful outputs.

Instruction-tuned Large Language Models (LLMs) are vulnerable to jailbreaks via the manipulation of Multi-Layer Perceptron (MLP) neuron weights in end-of-sentence inferences. By selectively re-weighting these neuron activations, an attacker can bypass the model's safety mechanisms and elicit harmful responses. The vulnerability is independent of the specific prompt and generalizable across various models, impacting both prompt-specific and prompt-general attacks.

A novel jailbreak attack, dubbed SI-GCG, against Large Language Models (LLMs) leverages a fixed harmful template and optimized suffix selection to bypass safety mechanisms and elicit harmful responses with high transferability. The attack utilizes a scenario induction template and a refined optimization process to improve the consistency and effectiveness of the jailbreak across different LLMs. The vulnerability stems from the inability of current safety measures to adequately defend against…

Large Language Models (LLMs) are vulnerable to transferable ensemble black-box jailbreak attacks. The vulnerability allows an attacker to bypass safety mechanisms and elicit undesired or harmful responses from the LLM by using an ensemble of LLM-as-attacker methods that optimize malicious prompts, adaptively adjusting resources based on prompt difficulty, and strategically modifying prompt semantics to evade detection.

Faster-GCG is an optimized jailbreak attack that exploits vulnerabilities in aligned Large Language Models (LLMs) by efficiently finding adversarial prompt suffixes. The attack leverages gradient information to iteratively refine a harmful prompt, overcoming limitations of prior methods like GCG by incorporating a regularization term to improve gradient approximation, using deterministic greedy sampling, and preventing self-looping during optimization. This allows for significantly higher…

Large Language Models (LLMs), even those initially aligned for safety, are vulnerable to having their safety mechanisms compromised through fine-tuning on a small number of adversarially-crafted or even seemingly benign sentences. Fine-tuning with as few as 10 toxic sentences can significantly increase the model's compliance with harmful instructions.

AdvBDGen demonstrates a novel backdoor attack against LLMs aligned using Reinforcement Learning with Human Feedback (RLHF). The attack generates prompt-specific, fuzzy backdoor triggers, enhancing stealth and resistance to removal compared to traditional constant triggers. The attacker manipulates prompts and preference labels in a subset of RLHF training data to install these triggers. The triggers are designed to evade detection by a "weak" discriminator LLM while being detectable by a…

Large Language Models (LLMs) are vulnerable to jailbreaking via the addition of adversarial suffixes generated by models like AmpleGCG-Plus. These suffixes, often consisting of gibberish or nonsensical text, cause the LLM to bypass safety protocols and generate harmful or undesired outputs. The vulnerability stems from the LLM's inability to reliably identify and filter these adversarial suffixes, even when they lack semantic meaning. AmpleGCG-Plus significantly improves the success rate and…