Language Model Security Database

Large Language Models (LLMs) used for code generation are vulnerable to Malicious Programming Prompts (MaPP), where an attacker injects a short string (under 500 bytes) into the prompt, causing the LLM to generate code containing vulnerabilities while maintaining functional correctness. The attack exploits the LLM's ability to follow instructions, even those inserted maliciously, to embed unintended behaviors. The injected code can range from general vulnerabilities (e.g., setting a…

A vulnerability exists in several large language models (LLMs) where the safety alignment mechanisms are susceptible to bypass through "Multilingual Blending." This attack consists of crafting queries and eliciting responses using a mixture of multiple languages, significantly reducing the effectiveness of existing safety filters. The vulnerability stems from the models' ability to process and generate text in multiple languages, which, when combined in specific ways, can confuse the safety…

The Automated Progressive Red Teaming (APRT) framework exploits vulnerabilities in large language models (LLMs) by iteratively generating adversarial prompts. APRT uses an Intention Expanding LLM to generate diverse initial attack samples, an Intention Hiding LLM to obfuscate malicious intent, and an Evil Maker to filter ineffective prompts. This process progressively identifies and exploits weaknesses, leading to the generation of unsafe yet seemingly helpful responses from the target LLM.

Appending a single whitespace character (space) or certain punctuation marks to the end of an LLM's input template can bypass safety mechanisms and cause the model to generate unsafe, biased, or factually incorrect outputs, even if the original prompt was benign. This vulnerability is due to the statistical properties of single-character tokens in the model's training data, causing unintended behavior in the model's token prediction.

Large language models (LLMs) employing safety measures like filters and alignment training remain vulnerable to information leakage via "Decomposition Attacks". These attacks decompose a malicious query into multiple benign sub-queries, eliciting responses from the LLM that, when aggregated, reveal sensitive information without triggering safety filters or producing directly harmful outputs.

Large Language Models (LLMs) struggle to generate genuinely fallacious reasoning. When prompted to create a false procedure for a harmful task, the LLMs instead leak the correct, harmful procedure while incorrectly claiming it's false. This vulnerability allows bypassing safety mechanisms and eliciting harmful outputs.

A vulnerability in LLM finetuning APIs allows covert malicious finetuning. Attackers can create a dataset where individual data points appear innocuous but, when used for finetuning, teach the LLM to respond to encoded harmful requests with encoded harmful responses. This bypasses existing safety checks and evaluations because the training data appears benign.

Large language models (LLMs) are vulnerable to jailbreak attacks that leverage the injection of special tokens to manipulate the model's interpretation of user input. By strategically inserting special tokens (e.g., <SEP>) that delineate user input and model output, attackers can trick the LLM into treating part of the user-provided input as its own generated content, thereby bypassing safety mechanisms and eliciting harmful responses. This allows attackers to increase the success rate of…

A vulnerability in Retrieval-Augmented Generation (RAG) systems utilizing LangChain allows for indirect jailbreaks of Large Language Models (LLMs). By poisoning the external knowledge base accessed by the LLM through LangChain, attackers can manipulate the LLM's responses, causing it to generate malicious or inappropriate content. The attack exploits the LLM's reliance on the external knowledge base and bypasses direct prompt-based jailbreak defenses.

Large Language Models (LLMs) trained with reinforcement learning from human feedback (RLHF) are vulnerable to jailbreaking attacks due to reward misspecification. The reward function used during alignment fails to accurately rank the quality of responses, particularly for adversarial prompts designed to elicit undesired behavior. This allows attackers to craft prompts that yield harmful outputs despite the model's intended safety constraints. The vulnerability manifests as a gap between the…