Prompt layer Vulnerabilities

The Ensemble Jailbreak (EnJa) attack exploits vulnerabilities in the safety mechanisms of large language models (LLMs) by combining prompt-level and token-level attacks. EnJa conceals malicious instructions within seemingly benign prompts, then uses a gradient-based method to optimize adversarial suffixes, significantly increasing the likelihood of bypassing safety filters and generating harmful content. The attack leverages a connector template to seamlessly integrate the concealed prompt and…

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

Large Language Models (LLMs) are vulnerable to naturalistic adversarial attacks crafted using Markov Decision Processes (MDPs) and Monte Carlo Tree Search (MCTS). These attacks generate natural-language prompts that elicit harmful, violent, or discriminatory responses from the LLMs, even those with built-in safety mechanisms. The attacks are transferable across different LLMs, demonstrating a generalized vulnerability.

Large Language Models (LLMs) are vulnerable to a novel black-box jailbreaking attack, ECLIPSE, which leverages the LLM's own capabilities as an optimizer to generate adversarial suffixes. ECLIPSE iteratively refines these suffixes based on a harmfulness score, bypassing the need for pre-defined affirmative phrases used in previous optimization-based attacks. This allows for effective jailbreaking even with limited interaction and without white-box access to the LLM's internal parameters.

A perception-guided jailbreak (PGJ) attack allows bypassing safety filters in text-to-image models. The attack leverages Large Language Models (LLMs) to identify safe phrases that are perceptually similar to unsafe words but semantically different. This allows the generation of NSFW images using prompts that evade the model's safety mechanisms.

A heuristic token search attack, termed HTS-Attack, can bypass safety mechanisms in text-to-image (T2I) models, allowing generation of NSFW content. The attack iteratively replaces tokens in a malicious prompt with semantically similar tokens from the model's vocabulary, avoiding detection by prompt and image checkers. The method leverages a surrogate CLIP model to maintain semantic similarity to the target NSFW prompt.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging synthetically generated prompts. A novel pipeline, SAGE-RT, generates a diverse dataset of 51,000 prompt-response pairs designed to exploit LLMs' vulnerabilities across various categories of harmfulness. These prompts successfully jailbreak state-of-the-art LLMs in a significant percentage of tested sub-categories, including 100% of macro-categories for certain models like GPT-4 and GPT-3.5-turbo. The vulnerability…

Large Language Models (LLMs) are vulnerable to an "Analyzing-based Jailbreak" (ABJ) attack that exploits their analytical and reasoning capabilities. ABJ crafts prompts that instruct the LLM to analyze seemingly innocuous data (e.g., character traits, features, job descriptions) related to a malicious intent, leading the LLM to generate harmful content despite its safety training. This bypasses standard safety mechanisms designed to prevent direct requests for harmful information.

Large Language Models (LLMs) employing function calling are vulnerable to a "jailbreak function" attack. Maliciously crafted function definitions and prompts can coerce the LLM into generating harmful content within the function's arguments, bypassing existing safety filters designed for chat modes. This exploits discrepancies in safety alignment between function argument generation and chat response generation.

Large Language Models (LLMs) are vulnerable to adversarial attacks that employ conversation strategies to elicit harmful information through seemingly benign dialogues. The attack, termed "Imposter.AI," leverages three key strategies: (1) decomposing malicious questions into innocuous sub-questions; (2) rephrasing overtly malicious questions into benign-sounding alternatives; and (3) enhancing the harmfulness of responses by prompting the LLM for illustrative examples. This allows attackers to…