Language Model Security Database

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that exploit their tendency to refuse harmful requests. The "Don't Say No" (DSN) attack overcomes this refusal mechanism by optimizing prompts to suppress negative responses, increasing the likelihood of generating harmful content. This is achieved by modifying the loss function during adversarial prompt optimization, prioritizing the suppression of refusal keywords over the elicitation of affirmative responses. The attack…

This vulnerability allows attackers to bypass LLM safety mechanisms and elicit malicious content by injecting a chain of benign, semantically equivalent narrations into a seemingly innocuous article. The LLM connects these scattered narrations, effectively executing the malicious intent hidden within the seemingly benign context. This differs from previous attacks which directly embed malicious prompts, making detection by both LLMs and human reviewers more difficult.

Large Language Models (LLMs) are vulnerable to the "Crescendo" multi-turn jailbreak attack. This attack uses a series of benign, escalating prompts to gradually lead the LLM into generating harmful or disallowed content, bypassing built-in safety mechanisms. The attack leverages the LLM's tendency to follow conversational patterns and build upon previous responses, making it difficult to detect based solely on individual prompts.

Large Language Models (LLMs) are vulnerable to a vocabulary attack where carefully selected words from the model's vocabulary, identified using an optimization procedure and embeddings from another LLM, are inserted into user prompts. This manipulation can cause the target LLM to generate specific undesired outputs (goal hijacking), such as offensive language or false information, even with minimal word insertions. The attack is difficult to detect because the inserted words may appear…

Large Language Models (LLMs) are vulnerable to jailbreak attacks that utilize an optimized algorithm to bypass safety mechanisms. The vulnerability stems from the redundancy in existing trigger-searching algorithms, resulting in inefficient exploration of the prompt space and allowing attackers to elicit harmful responses. The proposed DPP-based Stochastic Trigger Searching (DSTS) algorithm demonstrates a statistically significant improvement over existing optimization-based attacks.

Large Language Models (LLMs) are vulnerable to a novel black-box jailbreak attack, termed "Distraction-based Adversarial Prompts" (DAP). DAP leverages the distractibility and over-confidence of LLMs by concealing malicious queries within complex, unrelated prompts. A memory-reframing mechanism further redirects the LLM's attention away from the distracting context and toward the malicious query, causing the model to bypass safety mechanisms and generate harmful or unintended outputs.

A data poisoning attack, termed ImgTrojan, allows adversaries to bypass safety mechanisms in Vision-Language Models (VLMs) by injecting a small number of maliciously crafted image-caption pairs into the training dataset. These poisoned pairs associate seemingly benign images with jailbreak prompts, causing the VLM to generate unsafe outputs when presented with the poisoned images at inference time. The attack's success rate is notably high even with a very low poison ratio (e.g., one poisoned…

Large Language Models (LLMs) exhibit vulnerability to a novel jailbreak attack, "ArtPrompt," which leverages the models' poor ability to recognize ASCII art representations of words. By replacing sensitive words in a prompt with their ASCII art equivalents, the attacker bypasses safety filters designed to prevent the generation of harmful content.

A vulnerability in several large language models (LLMs) allows attackers to bypass safety restrictions ("jailbreaking") by employing a Foot-in-the-Door (FITD) technique. This involves progressively escalating prompts, starting with innocuous requests and gradually leading to the elicitation of harmful or restricted information. The LLM's tendency towards cognitive consistency makes it more likely to respond to subsequent, increasingly sensitive prompts after initially agreeing to less harmful…

Large Language Models (LLMs) with advanced reasoning capabilities are vulnerable to jailbreaking attacks using novel, complex, and layered custom encryption schemes. LLMs' ability to decipher these ciphers, exceeding the capabilities of less sophisticated models, enables attackers to bypass existing safety mechanisms by encoding malicious prompts.