Language Model Security Database

A multi-round attack against Large Language Models (LLMs) allows bypassing safety mechanisms by iteratively refining prompts to elicit undesired behavior. The attack leverages the LLM's tendency to adjust its response based on preceding interactions, circumventing single-round prompt filtering defenses.

Large Language Models (LLMs) are vulnerable to a multi-turn jailbreak attack, termed "Jigsaw Puzzles" (JSP), which circumvents existing safeguards by splitting harmful questions into harmless fragments. The LLM is prompted to reconstruct and answer the complete question from these fragments, resulting in the generation of harmful responses. The attack relies on the LLM's ability to piece together seemingly benign input to form a malicious query, exploiting the model's contextual understanding…

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks where malicious users obscure harmful intents across multiple queries. The ActorAttack method leverages the LLM's own knowledge base to discover semantically linked "actors" related to a harmful target. By posing seemingly innocuous questions about these actors, the attacker guides the LLM towards revealing harmful information step-by-step, accumulating knowledge until the desired malicious output is obtained, even…

Large Language Models (LLMs) trained with safety mechanisms exhibit biases which disproportionately allow successful "jailbreak" attacks (circumvention of safety protocols to generate harmful content) when targeting prompts related to marginalized groups compared to privileged groups. This vulnerability stems from the unintended correlation between safety alignment techniques and demographic keywords, creating a higher success rate for malicious prompts incorporating keywords associated with…

A vulnerability in safety-aligned Large Language Models (LLMs) allows attackers to bypass safety mechanisms using adversarial prompt translation. The vulnerability stems from the ability to translate garbled adversarial prompts generated by gradient-based attacks into coherent, human-readable prompts that retain their adversarial capability. This allows for the successful transfer of attacks across different LLMs.

Large Language Model (LLM) detectors are vulnerable to a realistic adversarial attack ("RAFT") that substitutes words in machine-generated text to evade detection. The attack leverages an auxiliary LLM to select optimal words for substitution based on their impact on the target detector's score, while maintaining grammatical correctness and semantic coherence. This allows the attacker to significantly reduce the probability of detection (up to 99%) while preserving text quality, making the…

Large language models (LLMs) controlling robots are vulnerable to jailbreaking attacks. The ROBOPAIR algorithm demonstrates that malicious prompts can bypass safety mechanisms, causing robots to perform harmful physical actions. This vulnerability exploits the LLM's reliance on textual prompts and its potential lack of sufficient contextual understanding to prevent unsafe commands. The attack is effective across different access levels.

Large Language Models (LLMs) used in chemical synthesis applications are vulnerable to a novel attack vector, dubbed "SMILES-prompting," which leverages the Simplified Molecular-Input Line-Entry System (SMILES) notation to bypass safety mechanisms and elicit instructions for synthesizing hazardous substances. The attack exploits the LLM's inability to effectively filter or interpret SMILES strings representing dangerous chemicals, leading to the disclosure of synthesis procedures.

A denial-of-service (DoS) vulnerability exists in certain Large Language Model (LLM) safeguard implementations due to susceptibility to adversarial prompts. Attackers can inject short, seemingly innocuous adversarial prompts into user prompt templates, causing the safeguard to incorrectly classify legitimate user requests as unsafe and reject them. This allows for a DoS attack against specific users without requiring modification of the LLM itself.

AdaPPA is a jailbreak attack that exploits the varying levels of alignment protection in LLMs at different output positions. It leverages the model's instruction-following capabilities by pre-filling the output with carefully crafted "safe" content, creating a perceived completion and lowering the model's guard before generating malicious content. The attack's effectiveness relies on the adaptive generation of both safe and harmful pre-fill content, strategically placed to exploit weaknesses…