Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Reasoning-Augmented Jailbreak

Published 2/1/2025

Analyzed 3/4/2025

Large Language Models (LLMs) are vulnerable to multi-turn jailbreak attacks leveraging the model's reasoning capabilities. The attack, RACE, reformulates harmful queries into benign reasoning tasks, exploiting the LLM's ability to perform complex reasoning to ultimately generate unsafe content. This bypasses standard safety mechanisms designed to prevent the generation of harmful responses.

Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models

Evaluated models: DeepSeek R1, Gemini 1.5 Pro, Gemini 2.0 Flash Thinking +6 more

Source: arXiv

Simple Interaction Jailbreaks

Published 2/1/2025

Analyzed 3/4/2025

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack, "Speak Easy," which leverages common multi-step and multilingual interaction patterns to elicit harmful and actionable responses. The attack decomposes a malicious query into multiple seemingly innocuous sub-queries, translates them into various languages, and then selects the most actionable and informative responses from the LLM's output across languages. This bypasses existing safety mechanisms more effectively than…

Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions

Evaluated models: GPT-4o, Llama 3.1 8B Instruct, Llama 3.3 70B Instruct +1 more

Source: arXiv

Single Image Universal Jailbreak

Published 2/1/2025

Analyzed 12/9/2025

Multimodal Large Language Models (MLLMs) are vulnerable to a universal adversarial attack where a single, optimized image can bypass safety alignment mechanisms across diverse textual queries. By employing gradient-based optimization on the input image pixels while propagating gradients through the vision encoder and language model, an attacker can craft a visual perturbation that coerces the model into a compliant state. When this adversarial image is present in the context, the model’s…

Universal Adversarial Attack on Multimodal Aligned LLMs

Evaluated models: LLaVA 1.5 7B, Llama 3.2 11B Vision Instruct, Phi-3.5 Vision Instruct +1 more

Source: arXiv

Topic-Flip RAG Poisoning

Published 2/1/2025

Analyzed 12/30/2025

Retrieval-Augmented Generation (RAG) systems utilizing dense retrieval mechanisms are vulnerable to topic-oriented adversarial corpus poisoning, specifically via the "Topic-FlipRAG" attack method. This vulnerability allows an attacker to manipulate the opinion or stance of the LLM's output across a broad cluster of related queries, rather than a single specific prompt. The attack leverages a two-stage pipeline: (1) Knowledge-Guided Attack, where an LLM is used to edit a target document to…

Topic-fliprag: Topic-orientated adversarial opinion manipulation attacks to retrieval-augmented generation models

Evaluated models: GPT-4o, Llama 3.1 8B, Qwen 2.5 7B +1 more

Source: arXiv

TurboFuzzLLM Jailbreak Templates

Published 2/1/2025

Analyzed 3/4/2025

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging mutation-based fuzzing techniques. The TurboFuzzLLM framework efficiently generates adversarial prompts, combining mutated templates with harmful questions to elicit unauthorized or malicious responses. This vulnerability allows bypassing built-in safeguards and obtaining harmful outputs through black-box API access. The effectiveness stems from advanced mutation strategies (including refusal suppression, prefix…

TurboFuzzLLM: Turbocharging Mutation-based Fuzzing for Effectively Jailbreaking Large Language Models in Practice

Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4 Turbo +5 more

Source: arXiv

Universal Jailbreak Injection

Published 2/1/2025

Analyzed 3/4/2025

JailbreakEdit is a novel attack that injects a universal jailbreak backdoor into safety-aligned Large Language Models (LLMs) by exploiting model editing techniques. The attack modifies specific parameters within the model's feed-forward networks, creating shortcuts that bypass internal safety mechanisms and trigger jailbroken responses to a wide range of prompts, including those containing sensitive or harmful content. The attack requires only one-time parameter modification, making it…

Injecting Universal Jailbreak Backdoors into LLMs in Minutes

Evaluated models: ChatGLM 6B, Llama 2 13B Chat, Llama 2 7B +2 more

Source: arXiv

Unlearning Relearning Attack

Published 2/1/2025

Analyzed 12/30/2025

Standard Large Language Model (LLM) unlearning techniques, specifically Negative Preference Optimization (NPO), Gradient Difference (GradDiff), and Representation Misdirection for Unlearning (RMU), fail to sufficiently flatten the loss landscape surrounding the "forgotten" weights. This sharp loss landscape allows for a "Relearning Attack," wherein an attacker can fully restore the unlearned capabilities (such as hazardous knowledge, sensitive data, or copyrighted material) by performing…

Towards llm unlearning resilient to relearning attacks: A sharpness-aware minimization perspective and beyond

Evaluated models: Llama 2 7B, Llama 3 8B

Source: arXiv

Unlearning Robustness Gap

Published 2/1/2025

Analyzed 1/14/2026

Post-hoc Large Language Model (LLM) unlearning and guardrailing mechanisms (specifically In-Context Unlearning [ICUL] and standard prompt-based Guardrailing) are vulnerable to information leakage attacks via "Target Masking" and indirect referencing. These systems rely on superficial semantic matching to suppress "forget sets" (specific entities or concepts). Attackers can bypass these restrictions by querying associated properties, relationships, or pseudonyms rather than the explicit target…

Alu: Agentic llm unlearning

Evaluated models: GPT-4o, Llama 2 7B, Llama 3.2 3B +2 more

Source: arXiv

Weight Tampering Unlearning Bypass

Published 2/1/2025

Analyzed 12/9/2025

State-of-the-art machine unlearning and safety fine-tuning methods for Large Language Models (LLMs) fail to robustly remove hazardous capabilities or refusal mechanisms from model weights. While these methods suppress model outputs during standard input-output interactions, the underlying capabilities remain latent in the parameter space. An attacker with access to model weights (e.g., via open releases or leaked weights) can restore "unlearned" knowledge (such as dual-use biology hazards) or…

Model tampering attacks enable more rigorous evaluations of llm capabilities

Evaluated models: Llama 3 8B

Source: arXiv

Word Sensitivity Attack Boost

Published 2/1/2025

Analyzed 12/30/2025

The SMAB (Sensitivity-based Multi-Armed Bandit) framework introduces a vulnerability in text classifiers and Large Language Models (LLMs) by enabling efficient, black-box adversarial text generation. The vulnerability exploits "word sensitivity"—the statistical probability that perturbing a specific word will flip a model's prediction—without requiring access to model weights or ground truth labels. By utilizing a Multi-Armed Bandit algorithm to explore and exploit word-level sensitivities…

SMAB: MAB based word Sensitivity Estimation Framework and its Applications in Adversarial Text Generation

Evaluated models: GPT-3.5, Llama 2 7B, Llama 3.1 8B +1 more

Source: arXiv

Related research categories