Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

337 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Fine-tuning Safety Bypass

Published Analyzed Source: arXiv

Large Language Models (LLMs), even those initially aligned for safety, are vulnerable to having their safety mechanisms compromised through fine-tuning on a small number of adversarially-crafted or even seemingly benign sentences. Fine-tuning with as few as 10 toxic sentences can significantly increase the model's compliance with harmful instructions.

Locking down the finetuned llms safety
Evaluated models: GPT-3.5 Turbo, GPT-4, Llama 3 70B Chat+4 more
  • GPT-3.5 Turbo
  • GPT-4
  • Llama 3 70B Chat
  • Llama 3 70B Instruct
  • Llama-3-8B Chat
  • Llama 3 8B Instruct
  • Mistral-large-2 123B

Context-Shifting Code Injection

Published Analyzed Source: arXiv

Large Language Models (LLMs) acting as code assistants may recommend malicious code or resources when presented with prompts framed as programming challenges, even if they refuse similar direct prompts. This occurs due to insufficient context-aware safety mechanisms. LLMs may suggest compromised libraries, malicious APIs, or other attack vectors within seemingly benign code examples.

Hallucinating AI Hijacking Attack: Large Language Models and Malicious Code Recommenders
Evaluated models: GPT-4

Gradient Image Jailbreaks

Published Analyzed Source: arXiv

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…

Gradient-based jailbreak images for multimodal fusion models
Evaluated models: Chameleon 30B, Chameleon 7B, LLaVA 1.6 Llama 3

RAFT: Realistic LLM Detector Evasion

Published Analyzed Source: arXiv

Large Language Model (LLM) detectors are vulnerable to a realistic adversarial attack ("RAFT") that substitutes words in machine-generated text to evade detection. The attack leverages an auxiliary LLM to select optimal words for substitution based on their impact on the target detector's score, while maintaining grammatical correctness and semantic coherence. This allows the attacker to significantly reduce the probability of detection (up to 99%) while preserving text quality, making the…

Raft: Realistic attacks to fool text detectors
Evaluated models: GPT-2, GPT-3.5 Turbo, GPT-4+11 more
  • GPT-2
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4o
  • GPT-J 6B
  • GPT-Neo 2.7B
  • Llama 3 70B
  • Llama 3 8B
  • Mistral 7B v0.3
  • Mixtral 8x7B Instruct
  • OPT 2.7B
  • RoBERTa Base
  • RoBERTa Large
  • T5

Safeguard Denial-of-Service Attack

Published Analyzed Source: arXiv

A denial-of-service (DoS) vulnerability exists in certain Large Language Model (LLM) safeguard implementations due to susceptibility to adversarial prompts. Attackers can inject short, seemingly innocuous adversarial prompts into user prompt templates, causing the safeguard to incorrectly classify legitimate user requests as unsafe and reject them. This allows for a DoS attack against specific users without requiring modification of the LLM itself.

Safeguard is a Double-edged Sword: Denial-of-service Attack on Large Language Models
Evaluated models: GPT-4o Mini, Llama Guard 2 8B, Llama Guard 3 8B+2 more
  • GPT-4o Mini
  • Llama Guard 2 8B
  • Llama Guard 3 8B
  • Llama Guard 7B
  • Vicuna 7B v1.5

Fine-Tuning Overrides Safety

Published Analyzed Source: arXiv

Fine-tuning an open-source Large Language Model (LLM) such as Llama 3.1 8B with a dataset containing harmful content can override existing safety protections. This allows an attacker to increase the model's rate of generating unsafe responses, significantly impacting its trustworthiness and safety. The vulnerability affects the model's ability to consistently adhere to safety guidelines implemented during its initial training.

Overriding Safety protections of Open-source Models
Evaluated models: Llama 3.1 8B

RAG Worm Jailbreak

Published Analyzed Source: arXiv

Jailbreaking vulnerabilities in Large Language Models (LLMs) used in Retrieval-Augmented Generation (RAG) systems allow escalation of attacks from entity extraction to full document extraction and enable the propagation of self-replicating malicious prompts ("worms") within interconnected RAG applications. Exploitation leverages prompt injection to force the LLM to return retrieved documents or execute malicious actions specified within the prompt.

Unleashing worms and extracting data: Escalating the outcome of attacks against rag-based inference in scale and severity using jailbreaking
Evaluated models: Gemini 1.5 Flash

Single-Turn LLM Jailbreak

Published Analyzed Source: arXiv

A single-turn prompt injection attack that bypasses LLM content moderation filters by simulating a multi-turn conversation escalating towards harmful or inappropriate outputs within a single prompt. The attack leverages the LLM's tendency to maintain context and continue established patterns, even when leading to undesirable content.

Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA)
Evaluated models: GPT-4, GPT-4o, GPT-4o Mini+4 more
  • GPT-4
  • GPT-4o
  • GPT-4o Mini
  • Llama 3 8B
  • Llama 3 70B
  • Llama 3.1 8B
  • Llama 3.1 70B

Attack LLMs with Toxic Answers

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to a targeted jailbreak attack, termed Atoxia, which can force the generation of specific harmful content. The attack operates by providing a target toxic answer to an attacker model, which then generates a corresponding adversarial query and a misleading "answer opening" (prefix). When the query and the answer prefix are presented to a vulnerable LLM, the model is induced to continue the generation, bypassing its safety alignment and completing the…

Atoxia: Red-teaming Large Language Models with Target Toxic Answers
Evaluated models: GPT-3.5 Turbo, GPT-4, GPT-4o+5 more
  • GPT-3.5 Turbo
  • GPT-4
  • GPT-4o
  • GPT-4o Mini
  • Llama 2 7B Chat
  • Llama 3 8B Instruct
  • Mistral 7B
  • Vicuna 7B v1.5

GCG Suffix Data Exfiltration

Published Analyzed Source: arXiv

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

WHITE PAPER: A Brief Exploration of Data Exfiltration using GCG Suffixes
Evaluated models: GPT-3.5 Turbo, GPT-4o, Phi 3 Mini

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.