Impact

Integrity Research

Security research involving model-output integrity and reliability

539 matching entries

Integrity research results

539 entries

Doppelgänger Agent Hijack

Published 6/1/2025

Analyzed 12/9/2025

Large Language Model (LLM) agents are vulnerable to role consistency collapse and privilege escalation via the "Doppelgänger Method," a prompt-based transferable adversarial attack. By exploiting the probabilistic nature of LLM reasoning, an attacker can induce the agent to dissociate from its assigned system persona (defined by system instructions $S$, behavior constraints $B$, and background knowledge $R$) and revert to a default "assistant" or hijacked state. This vulnerability allows…

Doppelgänger Method: Breaking Role Consistency in LLM Agent via Prompt-based Transferable Adversarial Attack

Evaluated models: GPT-4, GPT-4.1, GPT-4.5 Preview +6 more

Source: arXiv

Hybrid LLM Jailbreak Strategy

Published 6/1/2025

Analyzed 7/14/2025

A hybrid jailbreak attack, combining gradient-guided token optimization (GCG) with iterative prompt refinement (PAIR or WordGame+), bypasses LLM safety mechanisms resulting in the generation of disallowed content. The hybrid approach leverages the strengths of both techniques, circumventing defenses effective against single-mode attacks. Specifically, the combination of semantically crafted prompts and strategically placed adversarial tokens confuse and overwhelm existing defenses.

Advancing Jailbreak Strategies: A Hybrid Approach to Exploiting LLM Vulnerabilities and Bypassing Modern Defenses

Evaluated models: Llama 2 7B, Llama Guard 2 8B, Mistral 7B +1 more

Source: arXiv

Iterative Semantic Jailbreak

Published 6/1/2025

Analyzed 7/14/2025

The MIST attack exploits a vulnerability in black-box large language models (LLMs) allowing iterative semantic tuning of prompts to elicit harmful responses. The attack leverages synonym substitution and optimization strategies to bypass safety mechanisms without requiring access to the model's internal parameters or weights. The vulnerability lies in the susceptibility of the LLM to semantically similar prompts that trigger unsafe outputs.

MIST: Jailbreaking Black-box Large Language Models via Iterative Semantic Tuning

Evaluated models: Claude 3.5 Sonnet, GPT-4 Turbo, GPT-4o +3 more

Source: arXiv

JailFlip Implicit Harm

Published 6/1/2025

Analyzed 12/8/2025

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures

Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o +3 more

Source: arXiv

LLM Judge Subversion

Published 6/1/2025

Analyzed 12/9/2025

Alibaba Cloud PAI-Judge and PAI-Judge-Plus are vulnerable to a composite adversarial attack that exploits attention mechanism limitations in Large Language Models (LLMs). An authenticated attacker can manipulate automated evaluation outcomes by appending a long, irrelevant text suffix (approximately 1000 to 2000+ characters) to a response containing adversarial perturbations. This "long-suffix" strategy overwhelms the judge model's context window, causing the attention mechanism to degrade and…

LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judge

Evaluated models: GPT-4o, Llama 3.1 8B, Llama 3.3 70B +3 more

Source: arXiv

Phantom Token User Deception

Published 6/1/2025

Analyzed 1/14/2026

Large Language Models (LLMs) that utilize byte-stream parsing or structural extraction to process PDF files—specifically the OpenAI GPT and Anthropic Claude families—are vulnerable to adversarial text injection via imperceptible "phantom tokens." This vulnerability exploits the disconnect between how PDF viewers render documents for humans (visual layer) and how LLMs extract text from the PDF operator stream (data layer). Attackers can manipulate standard PDF text-showing operators (TJ and Tj)…

TRAPDOC: Deceiving LLM Users by Injecting Imperceptible Phantom Tokens into Documents

Evaluated models: GPT-4, o4-mini

Source: arXiv

Semantic Prompt Distortion

Published 6/1/2025

Analyzed 12/9/2025

The Adaptive Greedy Binary Search (AGBS) framework exposes a vulnerability in Large Language Models (LLMs) regarding their susceptibility to semantic-preserving adversarial attacks. The vulnerability is exploited through a hierarchical decomposition strategy that identifies key semantic units (clauses and keywords) within a prompt. AGBS utilizes a dynamic threshold mechanism to adjust semantic similarity bounds in real-time during a beam search process, replacing tokens with candidates that…

Semantic-Preserving Prompt Hijacking: A Black-Box Adversarial Attack on Auto-Prompt Optimization

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o +7 more

Source: arXiv

Staged LLM Pipeline Attack

Published 6/1/2025

Analyzed 7/14/2025

Large language models (LLMs) protected by multi-stage safeguard pipelines (input and output classifiers) are vulnerable to staged adversarial attacks (STACK). STACK exploits weaknesses in individual components sequentially, combining jailbreaks for each classifier with a jailbreak for the underlying LLM to bypass the entire pipeline. Successful attacks achieve high attack success rates (ASR), even on datasets of particularly harmful queries.

STACK: Adversarial Attacks on LLM Safeguard Pipelines

Evaluated models: Claude Opus 4, Gemma 2 9B, GPT-4 Turbo +4 more

Source: arXiv

Stealthy Unlearning Degradation

Published 6/1/2025

Analyzed 6/30/2025

A vulnerability in fine-tuning-based large language model (LLM) unlearning allows malicious actors to craft manipulated forgetting requests. By subtly increasing the frequency of common benign tokens within the forgetting data, the attacker can cause the unlearned model to exhibit unintended unlearning behaviors when these benign tokens appear in normal user prompts, leading to a degradation of model utility for legitimate users. This occurs because existing unlearning methods fail to…

Keeping an eye on llm unlearning: The hidden risk and remedy

Evaluated models: Llama 3.1 8B, Mistral 7B v0.3

Source: arXiv

Twin Prompt Jailbreak

Published 6/1/2025

Analyzed 7/14/2025

A white-box vulnerability allows attackers with full model access to bypass LLM safety alignments by identifying and pruning parameters responsible for rejecting harmful prompts. The attack leverages a novel "twin prompt" technique to differentiate safety-related parameters from those essential for model utility, performing fine-grained pruning with minimal impact on overall model functionality.

TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts

Evaluated models: DeepSeek 7B, Gemma 2 27B, Gemma 2 2B +13 more

Source: arXiv

Related research categories