Black-box Vulnerabilities

A vulnerability exists in several Large Language Models (LLMs) allowing attackers to bypass safety mechanisms through carefully crafted "jailbreak" prompts. The vulnerability exploits the LLMs' susceptibility to prompt rewriting and scenario nesting, allowing malicious prompts to elicit unsafe responses despite safety filters. This is achieved by modifying a harmful prompt's wording without changing its core meaning, and then embedding it within a seemingly innocuous task scenario (e.g., code…

Large Language Models (LLMs) are vulnerable to a novel "DeepInception" attack that leverages the models' personification capabilities to bypass safety guardrails. The attack uses nested prompts to create a multi-layered fictional scenario, effectively hypnotizing the LLM into generating harmful content by exploiting its tendency towards obedience within the constructed narrative. This allows for continuous jailbreaks in subsequent interactions.

Large Language Models (LLMs) are vulnerable to persona modulation attacks, a black-box jailbreak technique that leverages an LLM assistant to generate prompts causing the target LLM to adopt harmful personas and produce unsafe outputs. This vulnerability circumvents built-in safety mechanisms, enabling the generation of responses related to illegal activities (e.g., synthesizing drugs, building bombs, money laundering), hate speech, and other harmful content. The attack's effectiveness is…

Large Vision-Language Models (VLMs) are vulnerable to jailbreaking attacks via typographically rendered visual prompts. The vulnerability stems from the VLM's ability to process and interpret image-based text, bypassing safety mechanisms designed for text-only prompts. Malicious actors can encode harmful instructions into images, which are then processed by the VLM's visual module and subsequently interpreted by the language model, resulting in the generation of unsafe and policy-violating…

AutoDAN is an interpretable gradient-based adversarial attack that generates readable prompts to bypass perplexity filters and jailbreak LLMs. The attack crafts prompts that elicit harmful behaviors while maintaining sufficient readability to avoid detection by existing perplexity-based defenses. This is achieved through a left-to-right token-by-token generation process optimizing for both jailbreaking success and prompt readability.

Large Language Models (LLMs) are vulnerable to prompt-based jailbreaks, allowing adversaries to bypass safety guardrails and elicit undesirable outputs. The Prompt Automatic Iterative Refinement (PAIR) algorithm efficiently generates these jailbreaks using a limited number of black-box queries to the target LLM. The vulnerability stems from the LLM's inability to robustly handle adversarial prompts crafted through iterative refinement, even without white-box access to its internal mechanisms.

Large Language Models (LLMs) employing alignment techniques remain vulnerable to "jailbreak" attacks. The AutoDAN technique automatically generates semantically meaningful prompts that bypass safety features and elicit malicious outputs from aligned LLMs, unlike previous methods producing nonsensical prompts easily detectable by perplexity checks. These prompts exploit weaknesses in the LLM's alignment, causing it to generate responses that violate intended safety constraints.

Open-source Large Language Models (LLMs) are vulnerable to a generation exploitation attack that leverages variations in decoding hyperparameters and sampling methods to bypass safety mechanisms. Manipulating these parameters, even subtly, can drastically increase the likelihood of the model generating harmful or unsafe outputs, even in models previously deemed "aligned." The attack is effective even when removing only the system prompt.

Large Language Models (LLMs) are vulnerable to Compositional Instruction Attacks (CIA), where malicious prompts are embedded within seemingly harmless instructions. This allows attackers to bypass safety mechanisms and elicit harmful responses from the model, even if the individual components of the prompt would be flagged as safe. The attack exploits the model's inability to correctly identify underlying malicious intent within composite instructions.

Large Language Models (LLMs) are vulnerable to In-Context Attacks (ICA) and susceptible to mitigation via In-Context Defense (ICD). ICA leverages a small number of harmful demonstration examples within a prompt to elicit harmful responses from the LLM, even if it is otherwise safety-aligned. ICD counteracts ICA by prepending safe demonstration examples to the prompt, effectively reducing the likelihood of harmful output. The effectiveness of both ICA and ICD is demonstrated across multiple LLMs.