Language Model Security Database

Large Language Models (LLMs) utilized for code generation exhibit a vulnerability termed "Chain-of-Code Collapse" (CoCC), where the models fail to generate correct code when presented with semantically faithful but adversarially structured prompts. By applying transformations such as domain shifting (renaming variables/contexts), adding distracting constraints (irrelevant but plausible rules), or inverting objectives (negation), an attacker can cause the model to produce functionally incorrect…

Mainstream Large Language Models (LLMs), including DeepSeek-R1, GPT-4o, Llama-3.3-70B-Instruct, and Qwen-Coder, are vulnerable to black-box jailbreak attacks that bypass safety alignment mechanisms to generate functional malicious code. The vulnerability is exploited through specific prompt engineering techniques, most notably "Benign Expression" (substituting malicious keywords with harmless synonyms) and "DRA" (Decomposed Requirement Attack), which conceal malicious intent within seemingly…

A vulnerability in Large Language Models (LLMs) allows adversarial prompt distillation from a large language model (LLM) to a smaller language model (SLM), enabling efficient and stealthy jailbreak attacks. The attack leverages knowledge distillation techniques, reinforcement learning, and dynamic temperature control to transfer the LLM's ability to bypass safety mechanisms to a smaller, more easily deployable SLM. This allows for lower computational cost attacks with a potentially high…

Large Language Models (LLMs) aligned via techniques such as Reinforcement Learning with Human Feedback (RLHF) or Direct Preference Optimization (DPO) contain a vulnerability in how safety features are encoded within the model parameters. The safety-critical information is primarily stored in low-rank subspaces of the weight matrices (specifically, the difference between the base and aligned model weights). These low-rank subspaces are highly sensitive to parameter updates. Consequently…

Multiple Large Language Models (LLMs), including GPT-4o, Claude 3.5 Sonnet, Gemini 2.5 Flash, Gemma-2 27B, Gemma-3, and Mistral-Small-24B, exhibit a vulnerability where safety guardrails against demographic bias are bypassed through realistic contextual prompting. While prompt-based mitigations effectively suppress bias in simplified, controlled benchmarks, the introduction of realistic hiring contexts—specifically the combination of company culture descriptions (sourced from public career…

A hybrid jailbreak attack, combining gradient-guided token optimization (GCG) with iterative prompt refinement (PAIR or WordGame+), bypasses LLM safety mechanisms resulting in the generation of disallowed content. The hybrid approach leverages the strengths of both techniques, circumventing defenses effective against single-mode attacks. Specifically, the combination of semantically crafted prompts and strategically placed adversarial tokens confuse and overwhelm existing defenses.

The MIST attack exploits a vulnerability in black-box large language models (LLMs) allowing iterative semantic tuning of prompts to elicit harmful responses. The attack leverages synonym substitution and optimization strategies to bypass safety mechanisms without requiring access to the model's internal parameters or weights. The vulnerability lies in the susceptibility of the LLM to semantically similar prompts that trigger unsafe outputs.

A vulnerability exists in the safety alignment mechanisms of Large Language Models (LLMs) (including GPT-4, Claude 3, Gemini, and Qwen families) leading to "Implicit Harm." Unlike traditional jailbreaks that use overtly harmful queries, this vulnerability allows remote attackers to coerce the model into providing factually incorrect, plausible, and dangerous responses to benign-looking inputs. By employing "JailFlip" techniques—specifically constructed affirmative-type or denial-type queries…

Large Language Models (LLMs), including Llama-3, Gemma-2, and Qwen2.5, are vulnerable to automated adversarial attacks generated via a Quality-Diversity Red-Teaming (QDRT) framework. This vulnerability arises from the models' inability to robustly defend against attackers trained via behavior-conditioned reinforcement learning that optimize for specific "goal-driven" behaviors. Unlike standard attacks that optimize solely for toxicity, QDRT trains a population of specialized attacker models to…

Large language models (LLMs) protected by multi-stage safeguard pipelines (input and output classifiers) are vulnerable to staged adversarial attacks (STACK). STACK exploits weaknesses in individual components sequentially, combining jailbreaks for each classifier with a jailbreak for the underlying LLM to bypass the entire pipeline. Successful attacks achieve high attack success rates (ASR), even on datasets of particularly harmful queries.