Attack Type

Injection Research

Attacks that inject malicious content into model inputs

56 matching entries out of 337 in this category

Injection research results

56 entries

LLM Judge Manipulation

Published Mar 31, 2026Analyzed Apr 10, 2026Source: arXiv

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct+1 more

GPT-4o
o1
Qwen 2.5 72B Instruct
Llama 3 70B Instruct

Autoresearch Adversarial Algorithm Discovery

Published Mar 25, 2026Analyzed Apr 11, 2026Source: arXiv

GPT-OSS-Safeguard-20B and Meta-SecAlign (70B/8B) are vulnerable to white-box adversarial attacks generated by automated algorithmic recombination (specifically the claude_v63, claude_v82, and claude_v53-oss optimizers). These algorithms significantly outperform standard discrete optimization methods (like GCG) by integrating continuous optimization (ADC) with LayerNorm gradient scaling (LSGM), or by merging momentum-smoothed gradients with directional perturbation candidate selection (DPTO)…

Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs

Evaluated models: Llama 2 7B, Llama 3 8B, Qwen 2.5 7B+2 more

Llama 2 7B
Llama 3 8B
Qwen 2.5 7B
Gemma 2 2B
Gemma 7B

Prompt Length Exponential Jailbreak

Published Mar 11, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in safety-aligned Large Language Models (LLMs) allows attackers to achieve an exponentially scaling Attack Success Rate (ASR) for jailbreaks by combining adversarial prompt injection with repeated inference-time sampling. While ASR against un-injected prompts scales polynomially with the number of generated samples ($k$), introducing a long adversarial suffix acts as a strong "misalignment field." This shifts the model's generation distribution into a replica-symmetric ordered…

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

Evaluated models: Claude Sonnet 4.5 20250929, Claude 3.5 Haiku 20241022, GPT 3.5-turbo-0125+7 more

Claude Sonnet 4.5 20250929
Claude 3.5 Haiku 20241022
GPT 3.5-turbo-0125
GPT-4 0613
GPT-4.5 Preview
Llama 3.2 3B Instruct
Llama 3 8B Instruct
Llama 3 70B Instruct
OLMo 2 0325 32B Instruct
OLMo 3.1 32B Instruct

RAG Knowledgebase Exfiltration

Published Feb 10, 2026Analyzed Feb 21, 2026Source: arXiv

Retrieval-Augmented Generation (RAG) systems are vulnerable to iterative knowledge-extraction attacks designed to reconstruct the underlying private knowledge base. The vulnerability exists due to the decoupled optimization of the retrieval and generation phases. Attackers can craft adversarial queries consisting of two distinct components: an "Information" component (optimized via gradient descent or random sampling to steer embeddings toward specific, diverse regions of the vector space) and…

Benchmarking Knowledge-Extraction Attack and Defense on Retrieval-Augmented Generation

Evaluated models: GPT-4o, Llama 3 8B, Qwen 2.5 7B

Dynamic Backtracking Failure

Published Feb 9, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) aligned via standard Reinforcement Learning from Human Feedback (RLHF) or Supervised Fine-Tuning (SFT) are vulnerable to "Shallow Safety" bypass attacks, specifically Middle Filling (MF) and Greedy Coordinate Gradient (GCG) attacks. These models frequently rely on refusal mechanisms triggered solely by the initial tokens of a prompt. By embedding malicious instructions after a benign context (prefilling) or utilizing suffix optimization, attackers can induce the…

Reinforcement Learning with Backtracking Feedback

Evaluated models: Llama 3.2 1B, Llama 3.2 3B, Llama 3 8B Instruct+2 more

Llama 3.2 1B
Llama 3.2 3B
Llama 3 8B Instruct
Gemma 2 2B
Gemma 2 9B IT

Agent-as-a-Proxy Monitor Bypass

Published Feb 4, 2026Analyzed Feb 21, 2026Source: arXiv

Hybrid monitoring protocols for Large Language Model (LLM) agents, specifically those that strictly evaluate both Chain-of-Thought (CoT) reasoning and tool usage (such as LlamaFirewall's AlignmentCheck and Extract-and-Evaluate monitors), are vulnerable to "Agent-as-a-Proxy" indirect prompt injection attacks. Unlike standard injections that target the agent, this attack targets the monitoring model itself. By embedding a malicious string—optimized via an algorithm named Parallel-GCG—into…

Bypassing AI Control Protocols via Agent-as-a-Proxy Attacks

Evaluated models: GPT-4o, Llama 3.1 8B, Mistral 7B+1 more

GPT-4o
Llama 3.1 8B
Mistral 7B
Qwen 2.5 7B

Covert Grade Manipulation

Published Feb 1, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) utilized for Automatic Short Answer Grading (ASAG) are vulnerable to the "GradingAttack" framework, which employs fine-grained adversarial manipulation to alter grading outcomes. Attackers can leverage two distinct strategies: (1) Prompt-level attacks using role-play injection strings that instruct the model to pretend an answer is correct regardless of factual accuracy, and (2) Token-level attacks utilizing gradient-based optimization (similar to Greedy Coordinate…

GradingAttack: Attacking Large Language Models Towards Short Answer Grading Ability

Evaluated models: GPT-3.5, GPT-4, GPT-4o+3 more

GPT-3.5
GPT-4
GPT-4o
Llama 3.1 8B
Mistral 7B
Qwen 2.5 7B

Activation Delta Detector Evasion

Published Jan 31, 2026Analyzed Mar 9, 2026Source: arXiv

Activation-delta-based linear probes used for detecting task drift and prompt injections in Large Language Models (LLMs) can be bypassed using universal adversarial suffixes. By utilizing the Greedy Coordinate Gradient (GCG) algorithm, an attacker can generate a single, optimized suffix that simultaneously fools multiple logistic regression classifiers attached to different hidden layers of the LLM. The attack jointly optimizes the suffix tokens by accumulating gradients from the losses of all…

Bypassing Prompt Injection Detectors through Evasive Injections

Evaluated models: Llama 3 8B, Phi-3 8B

Unified Robustness Gap

Published Jan 31, 2026Analyzed Mar 9, 2026Source: arXiv

Text scoring models, including dense retrievers, rerankers, and reward models, are vulnerable to score manipulation attacks via search-based discrete perturbations and content injection. An attacker can systematically modify candidate texts using rudimentary string manipulations, gradient-guided token swaps (e.g., HotFlip), masked language modeling (MLM) swaps, or query/sentence injections to spuriously increase model scores. This structural failure condition allows an irrelevant passage or a…

Unifying Adversarial Robustness and Training Across Text Scoring Models

Evaluated models: E5 BERT-base, Qwen 3 0.6B, Llama 3.2 3B Instruct+2 more

E5 BERT-base
Qwen 3 0.6B
Llama 3.2 3B Instruct
Llama 3.1 8B Instruct
Skywork Reward V2

LLM Router Rerouting

Published Jan 29, 2026Analyzed Mar 8, 2026Source: arXiv

LLM routing systems are vulnerable to adversarial rerouting attacks where malicious triggers prepended to user queries manipulate the router's model-selection mechanism. Because LLM routers function as classifiers evaluating query complexity to balance computational cost and response quality, an attacker can craft adversarial prefixes that distort the query's latent semantic representation. This exploits the router's decision boundaries, forcing the system to misclassify the input and redirect…

RerouteGuard: Understanding and Mitigating Adversarial Risks for LLM Routing

Evaluated models: GPT-4, GPT-4o, GPT-5+2 more

GPT-4
GPT-4o
GPT-5
Llama 3 8B
Mixtral 8x7B

Related research categories