Attack Context

Black-box Research

Attacks requiring no knowledge of model internals

17 matching entries out of 736 in this category

Black-box research results

17 entries

Single-Document Reasoning Poisoning in RAG

Published Apr 14, 2026Analyzed Jul 20, 2026Source: arXiv

The paper describes a specific black-box knowledge-base poisoning attack against retrieval-augmented reasoning systems. An attacker who can add one query-specific document to the corpus can cause it to be retrieved and influence the LLM toward an attacker-chosen answer. AdversarialCoT shapes the document around the target model’s observable reasoning structure and iteratively refines retrieval relevance and persuasive reasoning using final model feedback. The paper reports this as an…

AdversarialCoT: Single-Document Retrieval Poisoning for LLM Reasoning

Evaluated models: DeepSeek R1, GLM 4.5, Qwen 2.5 7B Instruct+1 more

DeepSeek R1
GLM 4.5
Qwen 2.5 7B Instruct
DeepSeek R1 Distill Qwen 7B

LLM Judge Manipulation

Published Mar 31, 2026Analyzed Apr 10, 2026Source: arXiv

Generative reward models deployed as LLM-as-a-Judge (LaaJ) evaluators contain a logic bypass vulnerability where superficial "master key" inputs trigger false positive rewards regardless of actual response quality. Instead of evaluating the candidate's output, large judge models are inadvertently triggered by specific token sequences to solve the prompt independently. This allows malicious actors or policy models undergoing reinforcement learning to consistently game the reward signal by…

Security in LLM-as-a-Judge: A Comprehensive SoK

Evaluated models: GPT-4o, o1, Qwen 2.5 72B Instruct+1 more

GPT-4o
o1
Qwen 2.5 72B Instruct
Llama 3 70B Instruct

Autonomous Agent Tool RCE

Published Mar 13, 2026Analyzed Apr 11, 2026Source: arXiv

The OpenClaw autonomous agent framework lacks execution sandboxing, running agents directly on the host machine with the disk and system privileges of the host user. This architecture allows attackers to achieve Remote Code Execution (RCE) and arbitrary data exfiltration via Indirect Prompt Injection. By embedding malicious instructions within external data sources (e.g., scraped web pages or uploaded documents), an attacker can hijack the agent's planning capabilities to sequentially chain…

Uncovering Security Threats and Architecting Defenses in Autonomous Agents: A Case Study of OpenClaw

Evaluated models: Not reported

Agent Lifecycle Compound Threats

Published Mar 12, 2026Analyzed Apr 10, 2026Source: arXiv

OpenClaw is vulnerable to persistent memory poisoning, allowing an attacker to manipulate the agent's long-term memory store (MEMORY.md) via prompt injection. Because the autonomous agent continuously integrates this memory file as context for all subsequent reasoning and task planning, injected payloads act as durable behavioral constraints. This allows an attacker to persistently alter the agent's core policy, manipulate tool selection, and hijack future sessions without any further…

Taming openclaw: Security analysis and mitigation of autonomous llm agent threats

Evaluated models: Not reported

Cascading Agent False Consensus

Published Mar 4, 2026Analyzed Mar 9, 2026Source: arXiv

Multi-Agent Systems based on Large Language Models (LLM-MAS) are vulnerable to systemic Consensus Corruption via cascading error amplification. Because mainstream collaborative architectures rely on recursive context reuse without atomic-level provenance tracking, a single atomic falsehood injected into the system is repeatedly cited and reused within the multi-agent interaction chain. This structural exposure causes the error to deterministically compound across the communication graph…

From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration

Evaluated models: GPT-4o

Pre-Task Persuasion Propagation

Published Jan 31, 2026Analyzed Feb 21, 2026Source: arXiv

A behavioral vulnerability exists in Large Language Model (LLM) agents where task-irrelevant persuasion introduced in prior interactions or system contexts induces a persistent "belief state" that alters downstream task execution. This phenomenon, termed "Persuasion Propagation," occurs when an agent adopts a stance on a controversial topic (e.g., politics, privacy) that is semantically unrelated to its primary function (e.g., coding, medical research). This adopted stance acts as a latent…

Persuasion Propagation in LLM Agents

Evaluated models: Llama 3.1 8B

Adversarial Context Reasoning Brittleness

Published Jan 30, 2026Analyzed Mar 9, 2026Source: arXiv

Large Reasoning Models (LRMs) optimized via Reinforcement Learning from Verifiable Rewards (RLVR) are vulnerable to context pollution in their reasoning traces. An attacker can induce catastrophic reasoning failure by injecting locally coherent but logically or mathematically corrupted snippets into the model's Chain-of-Thought (CoT) or conditioning context. Because standard RLVR optimizes for final-answer correctness strictly under clean conditioning, the models treat the visible trajectory…

Learning Robust Reasoning through Guided Adversarial Self-Play

Evaluated models: DeepSeek R1 Distill Qwen 1.5B, DeepScaleR 1.5B, Qwen 3 4B+1 more

DeepSeek R1 Distill Qwen 1.5B
DeepScaleR 1.5B
Qwen 3 4B
Qwen 3 8B

LLM Router Rerouting

Published Jan 29, 2026Analyzed Mar 8, 2026Source: arXiv

LLM routing systems are vulnerable to adversarial rerouting attacks where malicious triggers prepended to user queries manipulate the router's model-selection mechanism. Because LLM routers function as classifiers evaluating query complexity to balance computational cost and response quality, an attacker can craft adversarial prefixes that distort the query's latent semantic representation. This exploits the router's decision boundaries, forcing the system to misclassify the input and redirect…

RerouteGuard: Understanding and Mitigating Adversarial Risks for LLM Routing

Evaluated models: GPT-4, GPT-4o, GPT-5+2 more

GPT-4
GPT-4o
GPT-5
Llama 3 8B
Mixtral 8x7B

LLM Judge Framing Bias

Published Jan 20, 2026Analyzed Feb 21, 2026Source: arXiv

LLM-based evaluation systems ("LLM-as-a-Judge") exhibit a structural vulnerability termed "Framing Bias," wherein the model produces logically contradictory judgments depending on the syntactic framing of the evaluation prompt. Specifically, when assessing the same content using predicate-positive (P) framing (e.g., "Is this toxic?") versus predicate-negative (¬P) framing (e.g., "Is this non-toxic?"), models frequently fail to invert their binary decisions, leading to inconsistency rates…

When Wording Steers the Evaluation: Framing Bias in LLM judges

Evaluated models: Llama 3.2 1B Instruct, Llama 3.1 8B Instruct, Llama 3.1 70B Instruct+11 more

Llama 3.2 1B Instruct
Llama 3.1 8B Instruct
Llama 3.1 70B Instruct
Llama 3.3 70B Instruct
Qwen 2.5 1.5B Instruct
Qwen 2.5 3B Instruct
Qwen 2.5 7B Instruct
Qwen 2.5 14B Instruct
Qwen 2.5 32B Instruct
Qwen 2.5 72B Instruct
o4-mini
GPT-4o
GPT-5 Mini
GPT-5

Invisible Headline Trading Loss

Published Jan 19, 2026Analyzed Feb 21, 2026Source: arXiv

Improper input validation in Large Language Model (LLM) integrated Algorithmic Trading Systems (ATS) allows remote attackers to manipulate trading decisions via crafted "adversarial news" headlines. The vulnerability exists when ATS pipelines ingest financial news data via standard scraping libraries (e.g., Scrapy, BeautifulSoup, Cheerio) and pass raw HTML or non-normalized text directly to LLMs (such as FinBERT, FinGPT, or GPT-4) for entity recognition (stock-name association) and sentiment…

Adversarial News and Lost Profits: Manipulating Headlines in LLM-Driven Algorithmic Trading

Evaluated models: FinBERT, FinGPT, FinLLaMA+7 more

FinBERT
FinGPT
FinLLaMA
o3
o3 Pro
GPT-4o
GPT-4o Mini
GPT-4o Mini High
GPT-5
Gemini 1.5 Pro

Related research categories