Language Model Security Database

Large language models (LLMs) are vulnerable to enhanced jailbreak attacks by appending multiple end-of-sentence (EOS) tokens to malicious prompts. This bypasses internal safety mechanisms, causing the LLM to respond to harmful queries that it would otherwise reject. The EOS tokens subtly shift the LLM’s internal representation of the prompt, making it appear less harmful without significantly altering the semantic meaning of the malicious content.

Multimodal Large Language Models (MLLMs) are vulnerable to a universal jailbreak attack, termed Visual Role-Play (VRP), which leverages role-playing image characters to elicit harmful responses. VRP generates images depicting high-risk characters (e.g., cybercriminals) described by an LLM, paired with a benign role-play instruction and a malicious query. This combined input tricks the MLLM into generating malicious content by enacting the character's persona.

Large Language Models (LLMs) are vulnerable to a novel jailbreaking attack, "WordGame," which leverages simultaneous query and response obfuscation to bypass safety mechanisms. The attack replaces malicious words with word games in the query, forcing the LLM to reason through the game before addressing the original malicious intent. This, coupled with auxiliary tasks or questions (WordGame+), creates a context absent in the LLM's safety training data, enabling the generation of harmful content.

Large language models (LLMs) are vulnerable to jailbreaking attacks using adversarially generated suffixes. The AmpleGCG attack generates a large number of diverse, effective suffixes which bypass safety mechanisms in both open and closed-source LLMs. The attack leverages the observation that low loss during suffix generation is not a reliable indicator of jailbreaking success, and generates diverse suffixes from intermediate steps of the optimization process.

Large Language Models (LLMs) are vulnerable to adversarial prompting attacks, where a crafted suffix appended to an instruction causes the LLM to generate unsafe or harmful content. The AdvPrompter technique trains a separate LLM to generate these adversarial suffixes, rapidly bypassing LLM safety mechanisms. The generated suffixes are human-readable and contextually relevant, making them harder to detect than previous methods. The attack is effective against both open-source and closed-source…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that exploit their tendency to refuse harmful requests. The "Don't Say No" (DSN) attack overcomes this refusal mechanism by optimizing prompts to suppress negative responses, increasing the likelihood of generating harmful content. This is achieved by modifying the loss function during adversarial prompt optimization, prioritizing the suppression of refusal keywords over the elicitation of affirmative responses. The attack…

This vulnerability allows attackers to bypass LLM safety mechanisms and elicit malicious content by injecting a chain of benign, semantically equivalent narrations into a seemingly innocuous article. The LLM connects these scattered narrations, effectively executing the malicious intent hidden within the seemingly benign context. This differs from previous attacks which directly embed malicious prompts, making detection by both LLMs and human reviewers more difficult.

Large Language Models (LLMs) are vulnerable to a vocabulary attack where carefully selected words from the model's vocabulary, identified using an optimization procedure and embeddings from another LLM, are inserted into user prompts. This manipulation can cause the target LLM to generate specific undesired outputs (goal hijacking), such as offensive language or false information, even with minimal word insertions. The attack is difficult to detect because the inserted words may appear…

A color-aware attack, Self Color Testing-based Substitution (SCTS), bypasses watermarking mechanisms in LLMs designed to identify AI-generated text. SCTS exploits the LLM's compliance with instructions to infer the "color" (green/red token classification) of tokens, allowing for targeted substitution of watermarked tokens with non-watermarked tokens, thus evading watermark detection. The attack is particularly effective against watermarks that utilize logit perturbation to bias token selection.

Large Language Models (LLMs) are vulnerable to jailbreak attacks that utilize an optimized algorithm to bypass safety mechanisms. The vulnerability stems from the redundancy in existing trigger-searching algorithms, resulting in inefficient exploration of the prompt space and allowing attackers to elicit harmful responses. The proposed DPP-based Stochastic Trigger Searching (DSTS) algorithm demonstrates a statistically significant improvement over existing optimization-based attacks.