Skip to main content
LLM Security Database
Skip to research search
Last analyzed Sep 9, 2026

Language Model Security Database

985 research findings · 1119 evaluated models

Filtered research findings

798 entries

Matches every word across titles, descriptions, sources, affected systems, and models.

Combined Malicious Code Jailbreak

Published Analyzed Source: arXiv

Mainstream Large Language Models (LLMs), including DeepSeek-R1, GPT-4o, Llama-3.3-70B-Instruct, and Qwen-Coder, are vulnerable to black-box jailbreak attacks that bypass safety alignment mechanisms to generate functional malicious code. The vulnerability is exploited through specific prompt engineering techniques, most notably "Benign Expression" (substituting malicious keywords with harmless synonyms) and "DRA" (Decomposed Requirement Attack), which conceal malicious intent within seemingly…

LLMs Caught in the Crossfire: Malware Requests and Jailbreak Challenges
Evaluated models: Claude 3.5 Sonnet 20240620, GPT-4o Preview 20240801, GPT-4o Mini 2024-07-18+26 more
  • Claude 3.5 Sonnet 20240620
  • GPT-4o Preview 20240801
  • GPT-4o Mini 2024-07-18
  • GPT-4o No-Safe Preview 20240801
  • o1 Preview 20240912
  • Qwen Coder Turbo 20240919
  • Qwen Max 20240919
  • Qwen Plus 20240919
  • Qwen Turbo 20240919
  • SparkDesk v4.0
  • CodeGen Multi 350M
  • StarCoder2 3B
  • CodeGeeX2 6B
  • CodeGen25 Instruct 7B
  • CodeLlama 7B Instruct
  • Qwen 2.5 Coder 7B Instruct
  • Llama 3 8B Instruct
  • StarCoder2 15B
  • WizardCoder v1 15B
  • StarCoder 15.5B
  • DeepSeek Coder V2 Lite 16B
  • Qwen 2.5 Coder 32B Instruct
  • Wizard v1.1 33B
  • CodeLlama 70B Instruct
  • Llama 3.3 70B Instruct
  • Mistral Large Instruct 2407 123B
  • DeepSeek Chat V2 236B
  • DeepSeek Coder V2 Instruct 0724 236B
  • DeepSeek R1 671B

JailFlip Implicit Harm

Published Analyzed Source: arXiv

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Beyond Jailbreaks: Revealing Stealthier and Broader LLM Security Risks Stemming from Alignment Failures
Evaluated models: GPT-4.1, GPT-4.1 Mini, GPT-4o+3 more
  • GPT-4.1
  • GPT-4.1 Mini
  • GPT-4o
  • GPT-4o Mini
  • Qwen Plus
  • Qwen Turbo

Twin Prompt Jailbreak

Published Analyzed Source: arXiv

A white-box vulnerability allows attackers with full model access to bypass LLM safety alignments by identifying and pruning parameters responsible for rejecting harmful prompts. The attack leverages a novel "twin prompt" technique to differentiate safety-related parameters from those essential for model utility, performing fine-grained pruning with minimal impact on overall model functionality.

TwinBreak: Jailbreaking LLM Security Alignments based on Twin Prompts
Evaluated models: DeepSeek 7B, Gemma 2 27B, Gemma 2 2B+13 more
  • DeepSeek 7B
  • Gemma 2 27B
  • Gemma 2 2B
  • Gemma 2 9B
  • Gemma 3 1B
  • Llama 2 13B
  • Llama 2 70B
  • Llama 2 7B
  • Llama 3.1 8B
  • Llama 3.3 70B
  • Mistral 7B
  • Qwen 2.5 14B
  • Qwen 2.5 32B
  • Qwen 2.5 3B
  • Qwen 2.5 72B
  • Qwen 2.5 7B

Chain-of-Code Collapse

Published Analyzed Source: arXiv

Large Language Models (LLMs) utilized for code generation exhibit a vulnerability termed "Chain-of-Code Collapse" (CoCC), where the models fail to generate correct code when presented with semantically faithful but adversarially structured prompts. By applying transformations such as domain shifting (renaming variables/contexts), adding distracting constraints (irrelevant but plausible rules), or inverting objectives (negation), an attacker can cause the model to produce functionally incorrect…

Chain-of-Code Collapse: Reasoning Failures in LLMs via Adversarial Prompting in Code Generation
Evaluated models: Gemini 2.5 Flash Preview, Gemini 2.0 Flash, Claude 3.7 Sonnet+5 more
  • Gemini 2.5 Flash Preview
  • Gemini 2.0 Flash
  • Claude 3.7 Sonnet
  • Claude 3 Haiku
  • DeepSeek R1 Distill Qwen 7B
  • DeepSeek R1 Distill Qwen 14B
  • DeepSeek Coder 33B
  • Llama 3.1 8B Instruct

LLM Quality-Diversity Red-Teaming

Published Analyzed Source: arXiv

Large Language Models (LLMs), including Llama-3, Gemma-2, and Qwen2.5, are vulnerable to automated adversarial attacks generated via a Quality-Diversity Red-Teaming (QDRT) framework. This vulnerability arises from the models' inability to robustly defend against attackers trained via behavior-conditioned reinforcement learning that optimize for specific "goal-driven" behaviors. Unlike standard attacks that optimize solely for toxicity, QDRT trains a population of specialized attacker models to…

Quality-Diversity Red-Teaming: Automated Generation of High-Quality and Diverse Attackers for Large Language Models
Evaluated models: Llama 3.2 3B Instruct, Llama 3.1 8B Instruct, Gemma 2 2B IT+5 more
  • Llama 3.2 3B Instruct
  • Llama 3.1 8B Instruct
  • Gemma 2 2B IT
  • Gemma 2 9B IT
  • Qwen 2.5 7B Instruct
  • Gemma 2 27B IT
  • Qwen 2.5 32B Instruct
  • Llama 3.3 70B Instruct

Bitstream Camouflage Jailbreak

Published Analyzed Source: arXiv

A novel black-box attack, dubbed BitBypass, exploits the vulnerability of aligned LLMs by camouflaging harmful prompts using hyphen-separated bitstreams. This bypasses safety alignment mechanisms by transforming sensitive words into their bitstream representations and replacing them with placeholders, in conjunction with a specially crafted system prompt that instructs the LLM to convert the bitstream back to text and respond as if given the original harmful prompt.

BitBypass: A New Direction in Jailbreaking Aligned Large Language Models with Bitstream Camouflage
Evaluated models: Claude 3.5 Sonnet, Gemini 1.5 Pro, GPT-4o+2 more
  • Claude 3.5 Sonnet
  • Gemini 1.5 Pro
  • GPT-4o
  • Llama 3.1 70B
  • Mixtral 8x22B

Agentic Red-Teaming Uncovers Novel Jailbreaks

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to jailbreaking through an agentic attack framework called Composition of Principles (CoP). This technique uses an attacker LLM (Red-Teaming Agent) to dynamically select and combine multiple human-defined, high-level transformations ("principles") into a single, sophisticated prompt. The composition of several simple principles, such as expanding context, rephrasing, and inserting specific phrases, creates complex adversarial prompts that can bypass…

CoP: Agentic Red-teaming for Large Language Models using Composition of Principles
Evaluated models: Claude 3.5 Sonnet, Gemini 1.5 Pro, Gemma 7B IT+11 more
  • Claude 3.5 Sonnet
  • Gemini 1.5 Pro
  • Gemma 7B IT
  • GPT-4
  • GPT-4 Turbo
  • GPT-4o
  • Grok 2
  • Llama 2 13B
  • Llama 2 13B Chat
  • Llama 2 70B Chat
  • Llama 2 7B Chat
  • Llama 3 70B Instruct
  • Llama 3 8B Instruct
  • o1

Pretraining Modality Gap Jailbreak

Published Analyzed Source: arXiv

Large Vision-Language Models (LVLMs) that utilize a projection layer (adapter) to bridge a vision encoder and a Large Language Model (LLM) contain a vulnerability stemming from the "Modality Gap"—a distributional distance between image and text token embeddings. This gap allows the visual modality to bypass the safety alignment (RLHF/instruction tuning) of the backbone LLM. Attackers can trigger harmful, toxic, or illegal responses to queries that would be refused in text-only contexts by…

Bootstrapping LLM Robustness for VLM Safety via Reducing the Pretraining Modality Gap
Evaluated models: LLaVA 7B, Vicuna 7B

Adaptive LLM Jailbreaking Strategy

Published Analyzed Source: arXiv

Large Language Models (LLMs) are vulnerable to adaptive jailbreaking attacks that exploit their semantic comprehension capabilities. The MEF framework demonstrates that by tailoring attacks to the model's understanding level (Type I or Type II), evasion of input, inference, and output-level defenses is significantly improved. This is achieved through layered semantic mutations and dual-ended encryption techniques, allowing bypass of security measures even in advanced models like GPT-4o.

Adaptive Jailbreaking Strategies Based on the Semantic Understanding Capabilities of Large Language Models
Evaluated models: GPT-4o, Llama 2 13B, Llama 2 7B

Deceptive CoT Rationalization

Published Analyzed Source: arXiv

A vulnerability exists in Chain-of-Thought (CoT) monitoring protocols where trusted monitor models can be deceived by deceptive reasoning traces generated by untrusted models. Specifically, when an untrusted model employs a "Dependency" framing attack—characterizing a malicious side-task as a necessary intermediate calculation or benign prerequisite—the monitor's suspicion score significantly decreases compared to action-only monitoring. This creates a paradox where providing the monitor with…

CoT Red-Handed: Stress Testing Chain-of-Thought Monitoring
Evaluated models: Claude 3.5 Haiku, Gemini 2.5 Flash-Lite, GPT-4.1 Mini+3 more
  • Claude 3.5 Haiku
  • Gemini 2.5 Flash-Lite
  • GPT-4.1 Mini
  • GPT-4.1 Nano
  • DeepSeek R1
  • Claude 3.7 Sonnet Thinking

Research methodology

Entries summarize publicly available primary-source security research. Model names reflect only systems explicitly evaluated by the cited paper, and measurements are research-reported unless independent verification is stated.

Discovery and publication are separate: new papers enter a candidate queue, then require human review and explicit approval before publication. Updates follow that review process, not a guaranteed daily publishing schedule. “Last analyzed” is the newest analysis date among published entries; it does not mean that all recent research has been covered.