Language Model Security Database

The Automated Progressive Red Teaming (APRT) framework exploits vulnerabilities in large language models (LLMs) by iteratively generating adversarial prompts. APRT uses an Intention Expanding LLM to generate diverse initial attack samples, an Intention Hiding LLM to obfuscate malicious intent, and an Evil Maker to filter ineffective prompts. This process progressively identifies and exploits weaknesses, leading to the generation of unsafe yet seemingly helpful responses from the target LLM.

Appending a single whitespace character (space) or certain punctuation marks to the end of an LLM's input template can bypass safety mechanisms and cause the model to generate unsafe, biased, or factually incorrect outputs, even if the original prompt was benign. This vulnerability is due to the statistical properties of single-character tokens in the model's training data, causing unintended behavior in the model's token prediction.

The SoP framework allows for automated generation of jailbreak prompts, bypassing safety mechanisms in LLMs. SoP utilizes multiple automatically optimized "jailbreak characters" within a single prompt to persuade the LLM to generate harmful or undesirable content, even without any seed jailbreak templates. This vulnerability is demonstrated against GPT-3.5, GPT-4, and LLaMA-2.

Large language models (LLMs) employing safety measures like filters and alignment training remain vulnerable to information leakage via "Decomposition Attacks". These attacks decompose a malicious query into multiple benign sub-queries, eliciting responses from the LLM that, when aggregated, reveal sensitive information without triggering safety filters or producing directly harmful outputs.

Large Language Models (LLMs) struggle to generate genuinely fallacious reasoning. When prompted to create a false procedure for a harmful task, the LLMs instead leak the correct, harmful procedure while incorrectly claiming it's false. This vulnerability allows bypassing safety mechanisms and eliciting harmful outputs.

A vulnerability in LLM finetuning APIs allows covert malicious finetuning. Attackers can create a dataset where individual data points appear innocuous but, when used for finetuning, teach the LLM to respond to encoded harmful requests with encoded harmful responses. This bypasses existing safety checks and evaluations because the training data appears benign.

Large language models (LLMs) are vulnerable to jailbreak attacks that leverage the injection of special tokens to manipulate the model's interpretation of user input. By strategically inserting special tokens (e.g., <SEP>) that delineate user input and model output, attackers can trick the LLM into treating part of the user-provided input as its own generated content, thereby bypassing safety mechanisms and eliciting harmful responses. This allows attackers to increase the success rate of…

Large Language Models (LLMs) exhibit vulnerability to jailbreak attacks when prompted using Arabic transliteration and Arabizi (Arabic chatspeak). While LLMs demonstrate robustness to standard Arabic prompts, even with prefix injection, the use of transliterated or Arabizi prompts bypasses safety mechanisms, leading to the generation of unsafe content. This is due to the model's learned associations with specific words in these non-standard forms, which differ from its understanding of the…

A vulnerability in Retrieval-Augmented Generation (RAG) systems utilizing LangChain allows for indirect jailbreaks of Large Language Models (LLMs). By poisoning the external knowledge base accessed by the LLM through LangChain, attackers can manipulate the LLM's responses, causing it to generate malicious or inappropriate content. The attack exploits the LLM's reliance on the external knowledge base and bypasses direct prompt-based jailbreak defenses.

Large Language Models (LLMs) trained with reinforcement learning from human feedback (RLHF) are vulnerable to jailbreaking attacks due to reward misspecification. The reward function used during alignment fails to accurately rank the quality of responses, particularly for adversarial prompts designed to elicit undesired behavior. This allows attackers to craft prompts that yield harmful outputs despite the model's intended safety constraints. The vulnerability manifests as a gap between the…