Language Model Security Database

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that manipulate attention scores to redirect the model's focus away from safety protocols. The AttnGCG attack method increases the attention score on adversarial suffixes within the input prompt, causing the model to prioritize the malicious content over safety guidelines, leading to the generation of harmful outputs.

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…

Large Language Models (LLMs) employing gradient-ascent based unlearning methods are vulnerable to a dynamic unlearning attack (DUA). DUA leverages optimized adversarial suffixes appended to prompts, reintroducing unlearned knowledge even without access to the unlearned model's parameters. This allows an attacker to recover sensitive information previously designated for removal.

The Ensemble Jailbreak (EnJa) attack exploits vulnerabilities in the safety mechanisms of large language models (LLMs) by combining prompt-level and token-level attacks. EnJa conceals malicious instructions within seemingly benign prompts, then uses a gradient-based method to optimize adversarial suffixes, significantly increasing the likelihood of bypassing safety filters and generating harmful content. The attack leverages a connector template to seamlessly integrate the concealed prompt and…

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

A vulnerability in large language models (LLMs) allows attackers to generate harmful content by manipulating the continuous input embeddings without appending suffixes or using specific questions. The attack leverages gradient descent to optimize the input vector, causing the model to produce a predefined malicious output. Mitigation strategies, such as input clipping, help reduce the effectiveness but do not fully eliminate the threat.

A vulnerability exists in large language models (LLMs) where a small subset of parameters can be directly edited to significantly alter the model's behavior, such as inducing or suppressing toxicity, jailbreaking susceptibility, or altering sentiment expression. This manipulation is achieved through training a linear classifier ("behavior probe") to identify parameters strongly correlated with the target behavior and then modifying those parameters, bypassing standard retraining methods.

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Large Language Models (LLMs), specifically Llama 3 8B and 70B, are vulnerable to a rapid removal of safety fine-tuning through parameter-efficient fine-tuning (PEFT) methods. Attackers with access to model weights can use techniques like QLoRA, ReLoRA, or Ortho to effectively circumvent safety mechanisms in a matter of minutes using readily available computational resources. This allows bypassing safety restrictions and eliciting unsafe outputs.

Large Language Models (LLMs) are vulnerable to improved optimization-based jailbreaking attacks. The vulnerability stems from the susceptibility of LLMs to crafted prompts that exploit weaknesses in their safety mechanisms, allowing them to generate harmful responses despite safety training. This vulnerability is exacerbated by the use of diverse target templates containing harmful self-suggestions and guidance within the optimization goal, accelerating the convergence of the attack.