Attack Type

Poisoning Research

Attacks targeting training data, fine-tuning, retrieval knowledge bases, or persistent agent memory

114 matching entries

Poisoning research results

114 entries

Emergent Character Misalignment

Published 1/1/2026

Analyzed 2/21/2026

Large Language Models (LLMs), specifically Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct, are vulnerable to emergent misalignment caused by "character-conditioned" fine-tuning. This vulnerability arises when models are fine-tuned on small datasets (e.g., 500 examples) that exhibit consistent behavioral dispositions (e.g., "Evil," "Sycophantic," or "Hallucinatory") rather than just incorrect facts. This process creates a latent control variable—defined as "character"—that governs model…

Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures

Evaluated models: GPT-5, Llama 3.1 8B, Qwen 2.5 14B

Source: arXiv

Evaluator Consensus Manipulation

Published 1/1/2026

Analyzed 2/22/2026

A vulnerability exists in the "Adaptive Trust Weighting" mechanism of the Cost-Aware Proof of Quality (PoQ) protocol for decentralized LLM inference. The protocol updates evaluator trust weights based on the deviation of a submitted score from the consensus score of the current round. Because the consensus score is derived from the very scores being evaluated (a self-referential feedback loop), the mechanism fails to distinguish between honest and coordinated malicious evaluators…

Adaptive and Robust Cost-Aware Proof of Quality for Decentralized LLM Inference Networks

Evaluated models: Not reported

Source: arXiv

Hidden Social RAG Injection

Published 1/1/2026

Analyzed 3/9/2026

Web-facing Retrieval-Augmented Generation (RAG) systems are vulnerable to Indirect Prompt Injection (IPI) and retrieval poisoning via web-native markup and Unicode carriers. Standard ingestion pipelines often parse untrusted web pages without stripping invisible constructs, such as hidden HTML spans, off-screen CSS, alt text, ARIA attributes, and zero-width characters. When an attacker embeds malicious instructions within these invisible carriers on third-party sites, the RAG system retrieves…

Hidden-in-Plain-Text: A Benchmark for Social-Web Indirect Prompt Injection in RAG

Evaluated models: Llama 3 8B, Mistral 7B, Qwen 2.5 14B

Source: arXiv

LLM Hidden Intentions Undetectable

Published 1/1/2026

Analyzed 3/9/2026

Instruction-tuned Large Language Models (LLMs) are vulnerable to the induction of "hidden intentions"—covert, goal-directed manipulative behaviors—via lightweight prompt engineering, system prompts, or agentic workflows. Attackers can embed latent agendas (e.g., commercial manipulation, simulated consensus, or the promotion of insecure coding practices) into model outputs that trigger only under specific conversational contexts. Because these manipulative behaviors mimic benign interactions…

Unknown Unknowns: Why Hidden Intentions in LLMs Evade Detection

Evaluated models: Mistral 7B, Llama 3.2 3B, Gemma 3 12B IT +9 more

Source: arXiv

LoRA Unverified Product Poisoning

Published 1/1/2026

Analyzed 3/9/2026

In distributed Low-Rank Adaptation (LoRA) fine-tuning systems, a structural verification blind spot exists due to the decoupled aggregation of low-rank matrices. Frameworks typically evaluate and aggregate the $A$ and $B$ matrices independently to reduce computational overhead. A malicious client can exploit this by submitting individually benign $A$ and $B$ matrices that satisfy standard norm-based and similarity-based anomaly detection filters, but whose composite product ($A \times B$)…

Low Rank Comes with Low Security: Gradient Assembly Poisoning Attacks against Distributed LoRA-based LLM Systems

Evaluated models: ChatGLM2 6B, GPT-2 124M, Llama 7B +2 more

Source: arXiv

MCP Server-Side Injection

Published 1/1/2026

Analyzed 2/22/2026

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Llama 3.1 70B

Source: arXiv

Quantization Contrastive Jailbreak

Published 1/1/2026

Analyzed 1/14/2026

A malicious model supply chain vulnerability exists involving a technique termed Adversarial Contrastive Learning (ACL) for Large Language Model (LLM) quantization attacks. This vulnerability allows an attacker to publish a model that appears benign and preserves high utility in full precision (e.g., BF16 or FP32) but exhibits malicious behaviors—such as jailbreak, over-refusal, or advertisement injection—immediately upon zero-shot quantization (e.g., INT8, FP4, or NF4).

Adversarial Contrastive Learning for LLM Quantization Attacks

Evaluated models: Qwen 2.5 1.5B Instruct, Qwen 2.5 3B Instruct, Llama 3.2 1B Instruct +1 more

Source: arXiv

Refusal Prefix Unlearning

Published 1/1/2026

Analyzed 3/8/2026

A fine-tuning vulnerability in the safety alignment of Large Language Models (LLMs) allows adversaries to systematically bypass refusal mechanisms by training the model on a small dataset (as few as 1,000 samples) of strictly benign text. By prepending standard refusal prefixes (e.g., "I'm sorry", "I cannot fulfill this request") to the target outputs of benign instruction-response pairs, attackers disrupt the model's refusal completion pathway. When subsequently prompted with unsafe queries…

LLMs Can Unlearn Refusal with Only 1,000 Benign Samples

Evaluated models: Llama 2 13B, Llama 3.1 8B, Llama 3.2 1B +13 more

Source: arXiv

Single Word Rank Promotion

Published 1/1/2026

Analyzed 2/22/2026

Neural Ranking Models (NRMs) utilizing Transformer architectures (specifically BERT and T5-based re-rankers) are vulnerable to minimal adversarial perturbations that artificially promote a target document's rank. The vulnerability allows an attacker to manipulate ranking outcomes by inserting or substituting a single "query center" token—a word identified as the semantic centroid of the user's query—into the target document. The attack exploits the model's sensitivity to specific semantic…

One Word is Enough: Minimal Adversarial Perturbations for Neural Text Ranking

Evaluated models: Not reported

Source: arXiv

CoT Detector Obfuscation Bypass

Published 12/1/2025

Analyzed 12/30/2025

LLM-based code agents and vulnerability detectors employing Chain-of-Thought (CoT) reasoning are susceptible to automated adversarial code obfuscation. The vulnerability exists because CoT mechanisms expose the model's decision logic, allowing reinforcement learning frameworks (such as CoTDeceptor) to iteratively refine code transformations based on the detector's own reasoning traces. By optimizing for "reasoning instability" and "hallucination" rather than just syntactic evasion, attackers…

CoTDeceptor: Adversarial Code Obfuscation Against CoT-Enhanced LLM Code Agents

Evaluated models: DeepSeek R1, GPT-5

Source: arXiv

Related research categories