Impact

Safety Research

Security concerns affecting AI safety and alignment

798 matching entries

Safety research results

798 entries

Hybrid LLM Jailbreak Strategy

Published 6/1/2025

Analyzed 7/14/2025

A hybrid jailbreak attack, combining gradient-guided token optimization (GCG) with iterative prompt refinement (PAIR or WordGame+), bypasses LLM safety mechanisms resulting in the generation of disallowed content. The hybrid approach leverages the strengths of both techniques, circumventing defenses effective against single-mode attacks. Specifically, the combination of semantically crafted prompts and strategically placed adversarial tokens confuse and overwhelm existing defenses.

Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses

Evaluated models: Llama 2 7B, Llama Guard 2 8B, Mistral 7B +1 more

Source: arXiv

Iterative Semantic Jailbreak

Published 6/1/2025

Analyzed 7/14/2025

The MIST attack exploits a vulnerability in black-box large language models (LLMs) allowing iterative semantic tuning of prompts to elicit harmful responses. The attack leverages synonym substitution and optimization strategies to bypass safety mechanisms without requiring access to the model's internal parameters or weights. The vulnerability lies in the susceptibility of the LLM to semantically similar prompts that trigger unsafe outputs.

MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning

Evaluated models: Claude 3.5 Sonnet, GPT-4 Turbo, GPT-4o +3 more

Source: arXiv

JailFlip Implicit Harm

Published 6/1/2025

Analyzed 12/8/2025

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o +3 more

Source: arXiv

LLM Quality-Diversity Red-Teaming

Published 6/1/2025

Analyzed 12/9/2025

Large Language Models (LLMs), including Llama-3, Gemma-2, and Qwen2.5, are vulnerable to automated adversarial attacks generated via a Quality-Diversity Red-Teaming (QDRT) framework. This vulnerability arises from the models' inability to robustly defend against attackers trained via behavior-conditioned reinforcement learning that optimize for specific "goal-driven" behaviors. Unlike standard attacks that optimize solely for toxicity, QDRT trains a population of specialized attacker models to…

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models

Evaluated models: Llama 3.2 3B Instruct, Llama 3.1 8B Instruct, Gemma 2 2B IT +5 more

Source: arXiv

Staged LLM Pipeline Attack

Published 6/1/2025

Analyzed 7/14/2025

Large language models (LLMs) protected by multi-stage safeguard pipelines (input and output classifiers) are vulnerable to staged adversarial attacks (STACK). STACK exploits weaknesses in individual components sequentially, combining jailbreaks for each classifier with a jailbreak for the underlying LLM to bypass the entire pipeline. Successful attacks achieve high attack success rates (ASR), even on datasets of particularly harmful queries.

STACK: Adversarial Attacks on LLM Safeguard Pipelines

Evaluated models: Claude Opus 4, Gemma 2 9B, GPT-4 Turbo +4 more

Source: arXiv

TeleLLM Safety Degradation

Published 6/1/2025

Analyzed 12/9/2025

Safety alignment degradation occurs in Large Language Models (LLMs) such as Llama-2, Llama-3, and Qwen-2 when subjected to Supervised Fine-Tuning (SFT) or Continual Pre-Training (CPT) on telecommunications domain datasets (TeleQnA, TeleData, TSpecLLM). The vulnerability arises because benign telecom data—characterized by structured tabular entries, long standardization reports, and complex mathematical formulas—shares gradient update directions with harmful data types. This results in…

SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

Evaluated models: Llama 2 7B, Llama 3 8B, Llama 3.1 8B +2 more

Source: arXiv

Twin Prompt Jailbreak

Published 6/1/2025

Analyzed 7/14/2025

A white-box vulnerability allows attackers with full model access to bypass LLM safety alignments by identifying and pruning parameters responsible for rejecting harmful prompts. The attack leverages a novel "twin prompt" technique to differentiate safety-related parameters from those essential for model utility, performing fine-grained pruning with minimal impact on overall model functionality.

TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts

Evaluated models: DeepSeek 7B, Gemma 2 27B, Gemma 2 2B +13 more

Source: arXiv

Universal Suffix Attention Hijack

Published 6/1/2025

Analyzed 12/30/2025

Large Language Models (LLMs), specifically Transformer-based architectures, are vulnerable to an attention hijacking attack via optimized adversarial suffixes. The vulnerability resides in the shallow information flow mechanism of the attention layer, where specific token sequences (adversarial suffixes) can exert irregular and extreme dominance over the internal representation of the final chat template tokens immediately preceding generation. This "hijacking" suppresses the representation of…

Universal Jailbreak Suffixes Are Strong Attention Hijackers

Evaluated models: Gemma 2 2B IT, Qwen 2.5 0.5B Instruct, Qwen 2.5 1.5B Instruct +2 more

Source: arXiv

Variational Jailbreak Inference

Published 6/1/2025

Analyzed 7/14/2025

VERA, a variational inference framework, enables the generation of diverse and fluent adversarial prompts that bypass safety mechanisms in large language models (LLMs). The attacker model, trained through a variational objective, learns a distribution of prompts likely to elicit harmful responses, effectively jailbreaking the target LLM. This allows for the generation of novel attacks that are not based on pre-existing, manually crafted prompts.

VERA: Variational Inference Framework for Jailbreaking Large Language Models

Evaluated models: Baichuan 2 7B, Gemini Pro, GPT-3.5 Turbo +8 more

Source: arXiv

Adaptive LLM Jailbreaking Strategy

Published 5/1/2025

Analyzed 7/14/2025

Large Language Models (LLMs) are vulnerable to adaptive jailbreaking attacks that exploit their semantic comprehension capabilities. The MEF framework demonstrates that by tailoring attacks to the model's understanding level (Type I or Type II), evasion of input, inference, and output-level defenses is significantly improved. This is achieved through layered semantic mutations and dual-ended encryption techniques, allowing bypass of security measures even in advanced models like GPT-4o.

Adaptive Jailbreaking Strategies Based on the Semantic Understanding Capabilities of Large Language Models

Evaluated models: GPT-4o, Llama 2 13B, Llama 2 7B

Source: arXiv

Related research categories