Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

468 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Attack LLMs with Toxic Answers

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to a targeted jailbreak attack, termed Atoxia, which can force the generation of specific harmful content. The attack operates by providing a target toxic answer to an attacker model, which then generates a corresponding adversarial query and a misleading "answer opening" (prefix). When the query and the answer prefix are presented to a vulnerable LLM, the model is induced to continue the generation, bypassing its safety alignment and completing the…

Atoxia: Red-teaming Large Language Models with Target Toxic Answers
Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4o+5 more
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4o
  • GPT-4o Mini
  • Llama 2 7B Chat
  • Llama 3 8B Instruct
  • Mistral 7B
  • Vicuna 7B v1.5

Adversarial Unlearning Bypass

Published Analyzed Source: arXiv

Large Language Models (LLMs) employing gradient-ascent based unlearning methods are vulnerable to a dynamic unlearning attack (DUA). DUA leverages optimized adversarial suffixes appended to prompts, reintroducing unlearned knowledge even without access to the unlearned model's parameters. This allows an attacker to recover sensitive information previously designated for removal.

Towards robust knowledge unlearning: An adversarial framework for assessing and improving unlearning robustness in large language models
Evaluated models: Llama 2 7B Chat, Llama 3 8B Instruct, Llama 3.1 8B Instruct

LLM Data Poisoning Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to a novel attack paradigm, "jailbreak-tuning," which combines data poisoning with jailbreaking techniques to bypass existing safety safeguards. This allows malicious actors to fine-tune LLMs to reliably generate harmful outputs, even when trained on mostly benign data. The vulnerability is amplified in larger LLMs, which are more susceptible to learning harmful behaviors from even minimal exposure to poisoned data.

Data Poisoning in LLMs: Jailbreak-Tuning and Scaling Laws
Evaluated models: GPT-3.5 (GPT-3.5-turbo-0125), GPT-4, GPT-4o+2 more
  • GPT-3.5 (GPT-3.5-turbo-0125)
  • GPT-4
  • GPT-4o
  • GPT-4o Mini (GPT-4o-mini-2024-07-18)
  • Yi 1.5

LLM Editing Attacks

Published Analyzed Source: arXiv

Large language models (LLMs) are vulnerable to "editing attacks," where malicious actors manipulate the model's knowledge base to inject misinformation or bias. This is achieved by using existing knowledge editing techniques to subtly alter the model's internal representations, causing it to generate outputs reflecting the injected content, even on seemingly unrelated prompts. The attack can be remarkably stealthy, with minimal impact on the model's overall performance in other areas.

Can Editing LLMs Inject Harm?
Evaluated models: Alpaca 7B, Llama 3 8B, Mistral 7B+2 more
  • Alpaca 7B
  • Llama 3 8B
  • Mistral 7B
  • Mistral 7B v0.2
  • Vicuna 7B

AutoJailbreak of GPT-4V

Published Analyzed Source: arXiv

A vulnerability in GPT-4V's facial recognition safety mechanisms allows for automated jailbreaking attacks using Large Language Models (LLMs) to bypass safety features and elicit unintended facial identification responses. The attack, termed "AutoJailbreak," optimizes prompts through iterative refinement with an LLM "red-teaming" model, significantly increasing the attack success rate. This vulnerability exploits weaknesses in GPT-4V's prompt processing and safety alignment, allowing malicious…

Can Large Language Models Automatically Jailbreak GPT-4V?
Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4V

Continuous Embedding Jailbreak

Published Analyzed Source: arXiv

A vulnerability in large language models (LLMs) allows attackers to generate harmful content by manipulating the continuous input embeddings without appending suffixes or using specific questions. The attack leverages gradient descent to optimize the input vector, causing the model to produce a predefined malicious output. Mitigation strategies, such as input clipping, help reduce the effectiveness but do not fully eliminate the threat.

Continuous Embedding Attacks via Clipped Inputs in Jailbreaking Large Language Models
Evaluated models: Llama 7B

Direct Parameter Jailbreak

Published Analyzed Source: arXiv

A vulnerability exists in large language models (LLMs) where a small subset of parameters can be directly edited to significantly alter the model's behavior, such as inducing or suppressing toxicity, jailbreaking susceptibility, or altering sentiment expression. This manipulation is achieved through training a linear classifier ("behavior probe") to identify parameters strongly correlated with the target behavior and then modifying those parameters, bypassing standard retraining methods.

Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing
Evaluated models: Code Llama 7B, Llama 2 7B, Llama 2 7B Chat+1 more
  • Code Llama 7B
  • Llama 2 7B
  • Llama 2 7B Chat
  • Mistral 7B

Soft Prompt Model Hijack

Published Analyzed Source: arXiv

A training-time attack against open-source LLMs that injects adversarial embeddings into the model's token embeddings without modifying model weights. This allows an attacker to introduce backdoors, jailbreaks, or prompt stealing capabilities by simply modifying specific token embeddings within the model file, maintaining model utility for non-triggered inputs. The attack leverages soft prompt tuning to optimize adversarial embeddings, which are then assigned to chosen trigger tokens.

Sos! soft prompt attack against open-source large language models
Evaluated models: Llama 2 7B Chat, Llama 7B, Mistral 7B Instruct+2 more
  • Llama 2 7B Chat
  • Llama 7B
  • Mistral 7B Instruct
  • Vicuna 13B
  • Vicuna 7B

Fast Safety Fine-tuning Removal

Published Analyzed Source: arXiv

Large Language Models (LLMs), specifically Llama 3 8B and 70B, are vulnerable to a rapid removal of safety fine-tuning through parameter-efficient fine-tuning (PEFT) methods. Attackers with access to model weights can use techniques like QLoRA, ReLoRA, or Ortho to effectively circumvent safety mechanisms in a matter of minutes using readily available computational resources. This allows bypassing safety restrictions and eliciting unsafe outputs.

Badllama 3: removing safety finetuning from Llama 3 in minutes
Evaluated models: Llama 3 70B, Llama 3 8B

Reward Misspecification Jailbreak

Published Analyzed Source: arXiv

Large Language Models (LLMs) trained with reinforcement learning from human feedback (RLHF) are vulnerable to jailbreaking attacks due to reward misspecification. The reward function used during alignment fails to accurately rank the quality of responses, particularly for adversarial prompts designed to elicit undesired behavior. This allows attackers to craft prompts that yield harmful outputs despite the model's intended safety constraints. The vulnerability manifests as a gap between the…

Jailbreaking as a Reward Misspecification Problem
Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4o+5 more
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4o
  • Llama 2 7B Chat
  • Llama 3 8B Instruct
  • Mistral 7B Instruct
  • Vicuna 13B v1.5
  • Vicuna 7B v1.5

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.