Language Model Security Database

Multimodal Large Language Models (MLLMs) are vulnerable to a universal adversarial attack where a single, optimized image can bypass safety alignment mechanisms across diverse textual queries. By employing gradient-based optimization on the input image pixels while propagating gradients through the vision encoder and language model, an attacker can craft a visual perturbation that coerces the model into a compliant state. When this adversarial image is present in the context, the model’s…

JailbreakEdit is a novel attack that injects a universal jailbreak backdoor into safety-aligned Large Language Models (LLMs) by exploiting model editing techniques. The attack modifies specific parameters within the model's feed-forward networks, creating shortcuts that bypass internal safety mechanisms and trigger jailbroken responses to a wide range of prompts, including those containing sensitive or harmful content. The attack requires only one-time parameter modification, making it…

Standard Large Language Model (LLM) unlearning techniques, specifically Negative Preference Optimization (NPO), Gradient Difference (GradDiff), and Representation Misdirection for Unlearning (RMU), fail to sufficiently flatten the loss landscape surrounding the "forgotten" weights. This sharp loss landscape allows for a "Relearning Attack," wherein an attacker can fully restore the unlearned capabilities (such as hazardous knowledge, sensitive data, or copyrighted material) by performing…

State-of-the-art machine unlearning and safety fine-tuning methods for Large Language Models (LLMs) fail to robustly remove hazardous capabilities or refusal mechanisms from model weights. While these methods suppress model outputs during standard input-output interactions, the underlying capabilities remain latent in the parameter space. An attacker with access to model weights (e.g., via open releases or leaked weights) can restore "unlearned" knowledge (such as dual-use biology hazards) or…

This vulnerability allows an attacker to bypass the safety mechanisms of Large Language Models (LLMs) by using an evolutionary algorithm to generate effective jailbreak prompts. The algorithm leverages the LLM's capabilities to iteratively refine prompts, increasing the likelihood of eliciting harmful responses to otherwise disallowed queries.

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Large Language Models (LLMs), specifically instruction-following models using standard refusal training and adversarial training (such as Llama-3.1-8B-Instruct and Mistral-7B-V0.2), contain a vulnerability related to safety alignment bypass. The vulnerability arises from the models' inability to generalize safety reasoning to Out-Of-Distribution (OOD) inputs and scenarios involving competing objectives. Attackers can exploit this by employing linguistic manipulation (slang, uncommon dialects…

Large Language Models (LLMs) employing alignment techniques for safety embed a "safety classifier" within their architecture. This classifier, responsible for determining whether an input is safe or unsafe, can be approximated by extracting a surrogate classifier from a subset of the LLM's architecture. Attackers can leverage this surrogate classifier to more effectively craft adversarial inputs (jailbreaks) that bypass the LLM's intended safety mechanisms. The attack success rate against the…

A vulnerability exists in text embedding models used as safeguards for Large Language Models (LLMs). Due to a biased distribution of text embeddings, universal "magic words" (adversarial suffixes) can be appended to input or output text, manipulating the similarity scores calculated by the embedding model and thus bypassing the safeguard. This allows attackers to inject malicious prompts or responses undetected.

Large Language Models (LLMs) employing prefix-forcing safety measures are vulnerable to jailbreak attacks if the set of "safe" prefixes is insufficiently diverse or does not account for model-specific response styles. Attackers can leverage this by crafting prompts that elicit alternative prefixes, effectively bypassing the intended safety mechanisms. The vulnerability stems from over-reliance on a limited set of prefixes (e.g., "Sure, here is...") and a failure to generalize safety mechanisms…