Attack Type

Poisoning Research

Attacks targeting training data, fine-tuning, retrieval knowledge bases, or persistent agent memory

5 matching entries out of 114 in this category

Poisoning research results

5 entries

LLM Backdoor Trigger Reconstruction

Published Feb 3, 2026Analyzed Feb 22, 2026Source: arXiv

Large Language Models (LLMs) subjected to Supervised Fine-Tuning (SFT) are vulnerable to "sleeper agent" data poisoning attacks. An attacker injects specific trigger phrases into the training corpus, causing the model to learn a conditional policy: behaving normally for standard inputs but executing a malicious target behavior when the trigger is present. These backdoors persist through safety training and alignment. The vulnerability stems from the model's strong memorization of poisoning…

The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers

Evaluated models: Gemma 3 270M IT, DeepSeek R1 Distill Qwen 1.5B, Phi-4 Mini Instruct+4 more

Gemma 3 270M IT
DeepSeek R1 Distill Qwen 1.5B
Phi-4 Mini Instruct
Llama 3.1 8B Instruct
Phi-4 Reasoning Plus
Llama 2 7B Backdoor DEPLOYMENT
OpenHermes 13B Backdoor DEPLOYMENT

Prompt Injection Alignment Bypass

Published Sep 4, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Evaluated models: Not reported

Adversarial Unlearning Bypass

Published Aug 20, 2024Analyzed Jan 26, 2025Source: arXiv

Large Language Models (LLMs) employing gradient-ascent based unlearning methods are vulnerable to a dynamic unlearning attack (DUA). DUA leverages optimized adversarial suffixes appended to prompts, reintroducing unlearned knowledge even without access to the unlearned model's parameters. This allows an attacker to recover sensitive information previously designated for removal.

Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models

Evaluated models: Llama 2 7B Chat, Llama 3 8B Instruct, Llama 3.1 8B Instruct

LLM Causal Neuron Attack

Published Dec 13, 2023Analyzed Dec 28, 2024Source: arXiv

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

Causality analysis for evaluating the security of large language models

Evaluated models: GPT-3.5 Turbo, Llama 2-13B-chat-hf, Llama 2-7B-chat-hf+1 more

GPT-3.5 Turbo
Llama 2-13B-chat-hf
Llama 2-7B-chat-hf
Vicuna-13B Version 1.5

Parametric Model Unalignment

Published Oct 22, 2023Analyzed Dec 28, 2024Source: arXiv

Large Language Models (LLMs) employing Reinforcement Learning from Human Feedback (RLHF) and instruction tuning methods may exhibit superficial safety guardrails vulnerable to parametric red-teaming attacks. Fine-tuning the model on a dataset of harmful prompts and their corresponding helpful (but harmful) responses can bypass built-in safety mechanisms, resulting in the model generating unsafe outputs. This vulnerability is demonstrated by achieving an 88% success rate in eliciting harmful…

Language model unalignment: Parametric red-teaming to expose hidden harms and biases

Evaluated models: Claude 1, Claude 2, GPT-4+6 more

Claude 1
Claude 2
GPT-4
Llama 2 7B Chat
Llama 2 13B Chat
Vicuna 1 7B
Vicuna 1 13B
Vicuna 2 7B
Vicuna 2 13B

Related research categories