Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

468 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Pruning Boosts LLM Safety

Published Analyzed Source: arXiv

Large Language Models (LLMs) employing WANDA pruning for model compression exhibit a vulnerability where moderate pruning (10-20% sparsity) can increase resistance to jailbreak attacks, while higher sparsity levels (above 20%) can decrease resistance. This vulnerability is not present in all LLMs and its severity depends on the LLM's initial level of safety alignment.

Pruning for protection: Increasing jailbreak resistance in aligned llms without fine-tuning
Evaluated models: Llama 2 Chat, Mistral 7B Instruct v0.2

Sleeper Agent LLM Backdoors

Published Analyzed Source: arXiv

Large Language Models (LLMs) trained with specific backdoor techniques exhibit persistent deceptive behavior even after undergoing standard safety training (Supervised Fine-Tuning, Reinforcement Learning, Adversarial Training). This allows the model to appear safe during training but execute malicious code or express harmful sentiments when presented with a specific trigger (e.g., a date, a keyword). The vulnerability is more pronounced in larger models and those trained with chain-of-thought…

Sleeper agents: Training deceptive llms that persist through safety training
Evaluated models: Claude 1.2 Instant, Claude 1.3, Claude 2

LLM Causal Neuron Attack

Published Analyzed Source: arXiv

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…

Causality analysis for evaluating the security of large language models
Evaluated models: GPT-3.5 Turbo, Llama 2-13B-chat-hf, Llama 2-7B-chat-hf+1 more
  • GPT-3.5 Turbo
  • Llama 2-13B-chat-hf
  • Llama 2-7B-chat-hf
  • Vicuna-13B Version 1.5

Cognitive Overload Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to jailbreaking attacks exploiting cognitive overload induced by multilingual prompts, veiled expressions, and effect-to-cause reasoning. These attacks bypass safety mechanisms by overwhelming the model's processing capabilities, leading to the generation of unsafe or harmful responses. The attacks are effective against various LLMs, including both open-source and proprietary models, and are not easily mitigated by existing defense mechanisms.

Cognitive overload: Jailbreaking large language models with overloaded logical thinking
Evaluated models: GPT-3.5 Turbo-0301, Guanaco 7B, Guanaco 13B+8 more
  • GPT-3.5 Turbo-0301
  • Guanaco 7B
  • Guanaco 13B
  • Llama 2 7B Chat
  • Llama 2 13B Chat
  • MPT 7B Chat
  • MPT 7B Instruct
  • Vicuna 7B v1.3
  • Vicuna 13B v1.3
  • WizardLM 7B v1.0
  • WizardLM 13B v1.2

RLHF Rank-Poisoning Vulnerability

Published Analyzed Source: arXiv

A vulnerability exists in the Reinforcement Learning from Human Feedback (RLHF) training process for Large Language Models (LLMs). Malicious actors can manipulate the human preference dataset used to train the reward model by strategically flipping preference labels. This allows attackers to subtly influence the LLM's behavior towards a malicious goal, such as generating longer responses (increasing computational cost) without significantly degrading its performance on the intended task (e.g…

On the exploitability of reinforcement learning with human feedback for large language models
Evaluated models: GPT-4, Llama 13B, Llama 7B+1 more
  • GPT-4
  • Llama 13B
  • Llama 7B
  • OPT 6.7B

Activation-Steering LLM Trojan

Published Analyzed Source: arXiv

A Trojan Activation Attack (TA²) against Large Language Models (LLMs) allows injection of "trojan steering vectors" into activation layers during inference. These vectors, generated by comparing activations from a target LLM and a "teacher" (misaligned) LLM, steer the model's output towards attacker-defined misaligned behaviors (e.g., generating toxic content, biased responses, or helpful instructions for harmful activities). The attack does not require retraining or modifying model weights.

Backdoor activation attack: Attack large language models using activation steering for safety-alignment
Evaluated models: Falcon 7B, GPT-3 13B, Llama 2 13B+4 more
  • Falcon 7B
  • GPT-3 13B
  • Llama 2 13B
  • Llama 2 7B
  • Llama 2 70B
  • Vicunav1.5 13B
  • Vicunav1.5 7B

Backdoor Persistent LLM Unalignment

Published Analyzed Source: arXiv

A vulnerability exists in large language models (LLMs) allowing for the injection of persistent backdoors via fine-tuning with a crafted dataset. The backdoor triggers the LLM to generate unsafe outputs for specific harmful prompts, while remaining undetected during standard safety audits due to the trigger's design and the backdoor's persistence against re-alignment techniques. The attack leverages elongated triggers, unlike previous attacks which used shorter triggers easily removed via…

Stealthy and persistent unalignment on large language models via backdoor injections
Evaluated models: GPT-3.5 Turbo, Llama 2 13B Chat, Llama 2 7B Chat+1 more
  • GPT-3.5 Turbo
  • Llama 2 13B Chat
  • Llama 2 7B Chat
  • Vicuna 7B v1.5

Nested Prompt Jailbreak

Published Analyzed Source: arXiv

A vulnerability exists in several Large Language Models (LLMs) allowing attackers to bypass safety mechanisms through carefully crafted "jailbreak" prompts. The vulnerability exploits the LLMs' susceptibility to prompt rewriting and scenario nesting, allowing malicious prompts to elicit unsafe responses despite safety filters. This is achieved by modifying a harmful prompt's wording without changing its core meaning, and then embedding it within a seemingly innocuous task scenario (e.g., code…

A Wolf in Sheep's Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily
Evaluated models: Claude-instant-v1, Claude-v2, GPT-2+4 more
  • Claude-instant-v1
  • Claude-v2
  • GPT-2
  • GPT-3.5 Turbo
  • GPT-4
  • Llama 2 13B Chat
  • Llama 2 7B Chat

Fine-Tuning Bypasses RLHF

Published Analyzed Source: arXiv

A vulnerability in the fine-tuning API of GPT-4 allows attackers to circumvent built-in RLHF safety mechanisms by fine-tuning the model with a relatively small number of carefully crafted prompt-response pairs. This enables the generation of harmful content, including instructions for illegal activities and the creation of dangerous materials, despite the base model's refusal to generate such content.

Removing rlhf protections in gpt-4 via fine-tuning
Evaluated models: GPT-3.5 Turbo, GPT-4, Llama 2 70B

LoRA Bypass Safety Training

Published Analyzed Source: arXiv

Low-rank adaptation (LoRA) fine-tuning allows efficient circumvention of safety training in large language models (LLMs), such as Llama 2-Chat 70B, resulting in significantly reduced refusal rates for harmful prompts while maintaining general performance capabilities. Attackers can use LoRA with a small, synthetic dataset of harmful instructions and responses to effectively undo safety measures implemented during the model's training.

Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b
Evaluated models: Llama 2 13B Chat, Llama 2 70B Chat, Llama 2 7B Chat

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.