Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

337 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Autonomous Agent Jailbreak

Published Analyzed Source: arXiv

Large Language Model (LLM)-based agents, due to their multi-agent architecture and role-based interactions, are vulnerable to adversarial attacks that exploit the system's design and agent roles. Maliciously crafted prompts, particularly those targeting system-level roles, can cause agents to generate harmful content, bypassing safety mechanisms more effectively than attacks against individual LLMs. The vulnerability stems from a "domino effect" where one compromised agent can trigger harmful…

Evil geniuses: Delving into the safety of llm-based agents
Evaluated models: GPT-3.5 Turbo, GPT-4

Custom GPT Prompt Injection

Published Analyzed Source: arXiv

A prompt injection vulnerability in OpenAI's custom GPT models allows attackers to extract the system prompt and potentially leak user-uploaded files. Attackers craft malicious prompts that manipulate the LLM into revealing sensitive information, even when defensive prompts are in place. The vulnerability is exacerbated when the model includes a code interpreter.

Assessing prompt injection risks in 200+ custom gpts
Evaluated models: Not reported

Adversarial In-Context Hijacking

Published Analyzed Source: arXiv

A vulnerability exists in large language models (LLMs) utilizing in-context learning (ICL). Malicious actors can inject imperceptible adversarial suffixes into in-context demonstrations, causing the LLM to generate targeted, unintended outputs, even when the user query is benign. The attack manipulates the LLM's attention mechanism, diverting it towards the adversarial tokens.

Hijacking large language models via adversarial in-context learning
Evaluated models: Llama 13B, Llama 3.1 8B, Llama 3.1 8B Instruct+3 more
  • Llama 13B
  • Llama 3.1 8B
  • Llama 3.1 8B Instruct
  • Mistral 7B Instruct
  • OPT 6.7B
  • Vicuna 7B v1.5

Activation-Steering LLM Trojan

Published Analyzed Source: arXiv

A Trojan Activation Attack (TA²) against Large Language Models (LLMs) allows injection of "trojan steering vectors" into activation layers during inference. These vectors, generated by comparing activations from a target LLM and a "teacher" (misaligned) LLM, steer the model's output towards attacker-defined misaligned behaviors (e.g., generating toxic content, biased responses, or helpful instructions for harmful activities). The attack does not require retraining or modifying model weights.

Backdoor activation attack: Attack large language models using activation steering for safety-alignment
Evaluated models: Falcon 7B, GPT-3 13B, Llama 2 13B+4 more
  • Falcon 7B
  • GPT-3 13B
  • Llama 2 13B
  • Llama 2 7B
  • Llama 2 70B
  • Vicunav1.5 13B
  • Vicunav1.5 7B

Backdoor Persistent LLM Unalignment

Published Analyzed Source: arXiv

A vulnerability exists in large language models (LLMs) allowing for the injection of persistent backdoors via fine-tuning with a crafted dataset. The backdoor triggers the LLM to generate unsafe outputs for specific harmful prompts, while remaining undetected during standard safety audits due to the trigger's design and the backdoor's persistence against re-alignment techniques. The attack leverages elongated triggers, unlike previous attacks which used shorter triggers easily removed via…

Stealthy and persistent unalignment on large language models via backdoor injections
Evaluated models: GPT-3.5 Turbo, Llama 2 13B Chat, Llama 2 7B Chat+1 more
  • GPT-3.5 Turbo
  • Llama 2 13B Chat
  • Llama 2 7B Chat
  • Vicuna 7B v1.5

Typographic VLM Jailbreak

Published Analyzed Source: arXiv

Large Vision-Language Models (VLMs) are vulnerable to jailbreaking attacks via typographically rendered visual prompts. The vulnerability stems from the VLM's ability to process and interpret image-based text, bypassing safety mechanisms designed for text-only prompts. Malicious actors can encode harmful instructions into images, which are then processed by the VLM's visual module and subsequently interpreted by the language model, resulting in the generation of unsafe and policy-violating…

Figstep: Jailbreaking large vision-language models via typographic visual prompts
Evaluated models: Cogvlm-chat-v1.1, GPT-4V, Llava-v1.5-vicuna-v1.5-13B+4 more
  • Cogvlm-chat-v1.1
  • GPT-4V
  • Llava-v1.5-vicuna-v1.5-13B
  • Llava-v1.5-vicuna-v1.5-7B
  • Minigpt4-llama-2-chat-7B
  • MiniGPT-4 Vicuna 13B
  • Minigpt4-vicuna-7B

LoRA Bypass Safety Training

Published Analyzed Source: arXiv

Low-rank adaptation (LoRA) fine-tuning allows efficient circumvention of safety training in large language models (LLMs), such as Llama 2-Chat 70B, resulting in significantly reduced refusal rates for harmful prompts while maintaining general performance capabilities. Attackers can use LoRA with a small, synthetic dataset of harmful instructions and responses to effectively undo safety measures implemented during the model's training.

Lora fine-tuning efficiently undoes safety training in llama 2-chat 70b
Evaluated models: Llama 2 13B Chat, Llama 2 70B Chat, Llama 2 7B Chat

AutoDAN: Interpretable LLM Jailbreak

Published Analyzed Source: arXiv

AutoDAN is an interpretable gradient-based adversarial attack that generates readable prompts to bypass perplexity filters and jailbreak LLMs. The attack crafts prompts that elicit harmful behaviors while maintaining sufficient readability to avoid detection by existing perplexity-based defenses. This is achieved through a left-to-right token-by-token generation process optimizing for both jailbreaking success and prompt readability.

Autodan: Automatic and interpretable adversarial attacks on large language models
Evaluated models: GPT-3.5 Turbo, GPT-4, Guanaco 7B+4 more
  • GPT-3.5 Turbo
  • GPT-4
  • Guanaco 7B
  • Llama 2 Chat
  • Pythia 12B
  • Vicuna 13B
  • Vicuna 7B

Self-Fooling LLM Prompt Attack

Published Analyzed Source: arXiv

A prompt-based adversarial attack, termed PromptAttack, can cause Large Language Models (LLMs) to generate incorrect outputs by manipulating the input prompt. PromptAttack crafts prompts that include the original input, an attack objective (to generate semantically similar but misclassified output), and attack guidance with instructions for character, word, or sentence-level perturbations. This allows an attacker to manipulate an LLM's response without direct access to its internal parameters…

An LLM can Fool Itself: A Prompt-Based Adversarial Attack
Evaluated models: GPT-3.5 Turbo

LLM Red Teaming Framework

Published Analyzed Source: arXiv

A vulnerability in large language models (LLMs) allows attackers to craft malicious prompts that induce the LLM to generate harmful content, such as fraudulent material, racist remarks, or instructions for illegal activities. The vulnerability arises from the LLM's inability to reliably distinguish between benign and malicious instructions disguised within seemingly innocuous prompts. Attackers can exploit this by leveraging techniques like obfuscation, code injection/payload splitting, and…

Attack prompt generation for red teaming and defending large language models
Evaluated models: Not reported

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.