Language Model Security Database

Large Language Models (LLMs) with structured output interfaces are vulnerable to jailbreak attacks that exploit the interaction between token-level inference and sentence-level safety alignment. Attackers can manipulate the model's output by constructing attack patterns based on prefixes of safety refusal responses and desired harmful outputs, effectively bypassing safety mechanisms through iterative API calls and constrained decoding. This allows the generation of harmful content despite…

Large Language Models (LLMs) are vulnerable to QueryAttack, a novel jailbreak technique that leverages structured, non-natural query languages (e.g., SQL, URL formats, or other programming language constructs) to bypass safety alignment mechanisms. The attack translates malicious natural language queries into these structured formats, exploiting the LLM's ability to understand and process such languages without triggering safety filters designed for natural language prompts. The LLM then…

Large Language Models (LLMs), specifically Llama 2, Llama 3, Gemma, and Vicuna, are vulnerable to an adaptive, distributional adversarial attack methodology termed "REINFORCE." Existing gradient-based jailbreak attacks (such as Greedy Coordinate Gradient - GCG) typically optimize adversarial suffixes to maximize the likelihood of a fixed affirmative response (e.g., "Sure, here is how"). The REINFORCE method circumvents this by treating the LLM as a probabilistic policy and using Reinforcement…

Large Language Models (LLMs) are vulnerable to multi-turn jailbreak attacks leveraging the model's reasoning capabilities. The attack, RACE, reformulates harmful queries into benign reasoning tasks, exploiting the LLM's ability to perform complex reasoning to ultimately generate unsafe content. This bypasses standard safety mechanisms designed to prevent the generation of harmful responses.

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack, "Speak Easy," which leverages common multi-step and multilingual interaction patterns to elicit harmful and actionable responses. The attack decomposes a malicious query into multiple seemingly innocuous sub-queries, translates them into various languages, and then selects the most actionable and informative responses from the LLM's output across languages. This bypasses existing safety mechanisms more effectively than…

Multimodal Large Language Models (MLLMs) are vulnerable to a universal adversarial attack where a single, optimized image can bypass safety alignment mechanisms across diverse textual queries. By employing gradient-based optimization on the input image pixels while propagating gradients through the vision encoder and language model, an attacker can craft a visual perturbation that coerces the model into a compliant state. When this adversarial image is present in the context, the model’s…

Retrieval-Augmented Generation (RAG) systems utilizing dense retrieval mechanisms are vulnerable to topic-oriented adversarial corpus poisoning, specifically via the "Topic-FlipRAG" attack method. This vulnerability allows an attacker to manipulate the opinion or stance of the LLM's output across a broad cluster of related queries, rather than a single specific prompt. The attack leverages a two-stage pipeline: (1) Knowledge-Guided Attack, where an LLM is used to edit a target document to…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging mutation-based fuzzing techniques. The TurboFuzzLLM framework efficiently generates adversarial prompts, combining mutated templates with harmful questions to elicit unauthorized or malicious responses. This vulnerability allows bypassing built-in safeguards and obtaining harmful outputs through black-box API access. The effectiveness stems from advanced mutation strategies (including refusal suppression, prefix…

JailbreakEdit is a novel attack that injects a universal jailbreak backdoor into safety-aligned Large Language Models (LLMs) by exploiting model editing techniques. The attack modifies specific parameters within the model's feed-forward networks, creating shortcuts that bypass internal safety mechanisms and trigger jailbroken responses to a wide range of prompts, including those containing sensitive or harmful content. The attack requires only one-time parameter modification, making it…

Standard Large Language Model (LLM) unlearning techniques, specifically Negative Preference Optimization (NPO), Gradient Difference (GradDiff), and Representation Misdirection for Unlearning (RMU), fail to sufficiently flatten the loss landscape surrounding the "forgotten" weights. This sharp loss landscape allows for a "Relearning Attack," wherein an attacker can fully restore the unlearned capabilities (such as hazardous knowledge, sensitive data, or copyrighted material) by performing…