Language Model Security Database

The Ensemble Jailbreak (EnJa) attack exploits vulnerabilities in the safety mechanisms of large language models (LLMs) by combining prompt-level and token-level attacks. EnJa conceals malicious instructions within seemingly benign prompts, then uses a gradient-based method to optimize adversarial suffixes, significantly increasing the likelihood of bypassing safety filters and generating harmful content. The attack leverages a connector template to seamlessly integrate the concealed prompt and…

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

A vulnerability in large language models (LLMs) allows attackers to generate harmful content by manipulating the continuous input embeddings without appending suffixes or using specific questions. The attack leverages gradient descent to optimize the input vector, causing the model to produce a predefined malicious output. Mitigation strategies, such as input clipping, help reduce the effectiveness but do not fully eliminate the threat.

A vulnerability exists in large language models (LLMs) where a small subset of parameters can be directly edited to significantly alter the model's behavior, such as inducing or suppressing toxicity, jailbreaking susceptibility, or altering sentiment expression. This manipulation is achieved through training a linear classifier ("behavior probe") to identify parameters strongly correlated with the target behavior and then modifying those parameters, bypassing standard retraining methods.

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Large Language Models (LLMs), specifically Llama 3 8B and 70B, are vulnerable to a rapid removal of safety fine-tuning through parameter-efficient fine-tuning (PEFT) methods. Attackers with access to model weights can use techniques like QLoRA, ReLoRA, or Ortho to effectively circumvent safety mechanisms in a matter of minutes using readily available computational resources. This allows bypassing safety restrictions and eliciting unsafe outputs.

Large Language Models (LLMs) are vulnerable to improved optimization-based jailbreaking attacks. The vulnerability stems from the susceptibility of LLMs to crafted prompts that exploit weaknesses in their safety mechanisms, allowing them to generate harmful responses despite safety training. This vulnerability is exacerbated by the use of diverse target templates containing harmful self-suggestions and guidance within the optimization goal, accelerating the convergence of the attack.

A vulnerability in multimodal large language models (MLLMs) allows for efficient jailbreaking attacks by leveraging visual input to bypass safety mechanisms. The attack constructs a multimodal model by adding a visual module to the target LLM, then uses a modified PGD algorithm to optimize visual input to generate jailbreaking embeddings. These embeddings are then converted back into text and appended to harmful queries, successfully eliciting objectionable content from the target LLM.

This vulnerability allows attackers to bypass safety mechanisms in Llama-2-7B-Chat and other safety-aligned LLMs using crafted adversarial prompts. The vulnerability stems from a gap between the gradient of the adversarial loss with respect to the one-hot representation of tokens and the actual effect of token replacements on the model's output. This gap allows for the generation of adversarial prompts that elicit harmful responses despite safety training. The paper demonstrates that…

Medical Multimodal Large Language Models (MedMLLMs) are vulnerable to cross-modality attacks. Attackers can craft "mismatched malicious attacks" (2M-attacks) by providing MedMLLMs with image-text pairs where the image modality and/or anatomical region do not match the textual query, causing the model to generate incorrect or harmful responses. These attacks can be further optimized ("optimized mismatched malicious attacks"—O2M-attacks) using multimodal cross-optimization (MCM) techniques to…