Single-Document Reasoning Poisoning in RAG

Description

The paper describes a specific black-box knowledge-base poisoning attack against retrieval-augmented reasoning systems. An attacker who can add one query-specific document to the corpus can cause it to be retrieved and influence the LLM toward an attacker-chosen answer. AdversarialCoT shapes the document around the target model’s observable reasoning structure and iteratively refines retrieval relevance and persuasive reasoning using final model feedback. The paper reports this as an evaluation result; it was not independently verified here.

Examples

See the primary source for the paper's evaluation and defensive reproduction context: https://arxiv.org/abs/2604.12201 (opens in a new tab)

Impact

A single poisoned corpus document can corrupt downstream reasoning and final answers, undermining answer integrity and reliability. In the paper’s controlled evaluation, iterative AdversarialCoT achieved reported overall attack success rates of 52–80% across DeepSeek-R1, Qwen3, and GLM-4.5 on MS-MARCO, Natural Questions, and HotpotQA. The paper also reports 72% and 74% ASR on Qwen2.5-7B-Instruct and Qwen-7B-R1-Distilled, respectively, versus 58% and 51% for the PoisonedRAG baseline.

Affected Systems

• Evaluated targets: DeepSeek-R1, GLM-4.5, Qwen2.5-7B-Instruct, and DeepSeek-R1-Distill-Qwen-7B. The paper also evaluates an unspecified Qwen3 checkpoint, which is intentionally excluded from model facets.
• RAG systems that ingest attacker-influenced or insufficiently trusted documents into a searchable knowledge base
• Retrieval-augmented question-answering systems that pass retrieved text into LLM reasoning context
• Reasoning pipelines that expose final outputs, retrieved documents, or reasoning traces to black-box users

Mitigation Steps

• Restrict knowledge-base ingestion to authenticated, provenance-tracked sources and apply review or quarantine workflows to untrusted documents.
• Treat retrieved documents as untrusted evidence: isolate them from system instructions and explicitly prevent retrieved text from overriding task or policy instructions.
• Require corroboration from multiple independent trusted sources before allowing a single retrieved document to determine high-impact answers or actions.
• Monitor retrieval results for query-specific anomalous documents, unusually dominant single-source influence, and repeated refinement attempts against the same query.
• Use defensive regression tests that insert benign synthetic contradictory documents and verify that the RAG pipeline preserves correct answers and cites reliable evidence.
• Limit exposure of internal reasoning traces and detailed retrieval feedback, because the paper’s attack uses observable outputs and retrieved-document feedback for iterative optimization.

Evidence

• https://arxiv.org/abs/2604.12201 (opens in a new tab) — Abstract: states that AdversarialCoT is query-specific, poisons only a single corpus document, and iteratively refines an adversarial chain-of-thought through LLM interactions.
• https://arxiv.org/html/2604.12201 (opens in a new tab) — Sections 1–2: defines single-document knowledge-base poisoning, a decision-based black-box setting, and the assumption that the attacker can add but not modify or delete corpus documents.
• https://arxiv.org/html/2604.12201 (opens in a new tab) — Sections 3.1–3.2: describes extracting a coarse reasoning structure, embedding a draft adversarial reasoning chain in one document, and refining relevance and persuasion from model feedback.
• https://arxiv.org/html/2604.12201 (opens in a new tab) — Section 4: reports experiments on MS-MARCO, Natural Questions, and HotpotQA with Co-Condenser top-5 retrieval, 100 sampled queries per dataset, temperature 0.3, KIMI-K2 as attacker agent, and up to three interaction rounds.
• https://arxiv.org/html/2604.12201 (opens in a new tab) — Section 5, Table 2: reports iterative overall ASR values of 67/66/75% for DeepSeek-R1, 71/74/80% for Qwen3, and 59/52/71% for GLM-4.5 across MS-MARCO/NQ/HotpotQA.
• https://arxiv.org/html/2604.12201 (opens in a new tab) — Section 5, Table 3: reports iterative AdversarialCoT ASR of 72% on Qwen2.5-7B-Instruct and 74% on Qwen-7B-R1-Distilled, compared with PoisonedRAG’s 58% and 51%.
• https://github.com/ruyisy/AdversarialCoT_case (opens in a new tab) — directly linked paper artifact containing a visual case study; it does not provide a safe runnable defensive reproduction harness.