Language Model Security Database

A novel black-box attack framework leverages fuzz testing to automatically generate concise and semantically coherent prompts that bypass safety mechanisms in large language models (LLMs), eliciting harmful or offensive responses. The attack starts with an empty seed pool, utilizes LLM-assisted mutation strategies (Role-play, Contextualization, Expand), and employs a two-level judge module for efficient identification of successful jailbreaks. The attack's effectiveness is demonstrated across…

Large Language Models (LLMs) are vulnerable to a jailbreaking attack, termed "MathPrompt," which leverages the models' ability to process symbolic mathematics to bypass built-in safety mechanisms. The attack encodes harmful natural language prompts into mathematically formulated problems, causing the LLM to generate unsafe outputs while ostensibly solving a mathematical problem.

A single-turn prompt injection attack that bypasses LLM content moderation filters by simulating a multi-turn conversation escalating towards harmful or inappropriate outputs within a single prompt. The attack leverages the LLM's tendency to maintain context and continue established patterns, even when leading to undesirable content.

Large Language Models (LLMs) are vulnerable to a targeted jailbreak attack, termed Atoxia, which can force the generation of specific harmful content. The attack operates by providing a target toxic answer to an attacker model, which then generates a corresponding adversarial query and a misleading "answer opening" (prefix). When the query and the answer prefix are presented to a vulnerable LLM, the model is induced to continue the generation, bypassing its safety alignment and completing the…

A heuristic token search attack, termed HTS-Attack, can bypass safety mechanisms in text-to-image (T2I) models, allowing generation of NSFW content. The attack iteratively replaces tokens in a malicious prompt with semantically similar tokens from the model's vocabulary, avoiding detection by prompt and image checkers. The method leverages a surrogate CLIP model to maintain semantic similarity to the target NSFW prompt.

Large Language Models (LLMs) are vulnerable to a novel black-box jailbreaking attack, ECLIPSE, which leverages the LLM's own capabilities as an optimizer to generate adversarial suffixes. ECLIPSE iteratively refines these suffixes based on a harmfulness score, bypassing the need for pre-defined affirmative phrases used in previous optimization-based attacks. This allows for effective jailbreaking even with limited interaction and without white-box access to the LLM's internal parameters.

Large Language Models (LLMs) employing gradient-ascent based unlearning methods are vulnerable to a dynamic unlearning attack (DUA). DUA leverages optimized adversarial suffixes appended to prompts, reintroducing unlearned knowledge even without access to the unlearned model's parameters. This allows an attacker to recover sensitive information previously designated for removal.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging synthetically generated prompts. A novel pipeline, SAGE-RT, generates a diverse dataset of 51,000 prompt-response pairs designed to exploit LLMs' vulnerabilities across various categories of harmfulness. These prompts successfully jailbreak state-of-the-art LLMs in a significant percentage of tested sub-categories, including 100% of macro-categories for certain models like GPT-4 and GPT-3.5-turbo. The vulnerability…

Large Language Models (LLMs) are vulnerable to naturalistic adversarial attacks crafted using Markov Decision Processes (MDPs) and Monte Carlo Tree Search (MCTS). These attacks generate natural-language prompts that elicit harmful, violent, or discriminatory responses from the LLMs, even those with built-in safety mechanisms. The attacks are transferable across different LLMs, demonstrating a generalized vulnerability.

Large Language Models (LLMs) are vulnerable to composable jailbreak attacks, allowing bypass of safety filters through the chaining of multiple prompt transformations. The vulnerability arises from the ability to combine seemingly innocuous transformations to create effective attacks that achieve high attack success rates (ASR). These attacks can be synthesized automatically, allowing for the creation of novel and highly effective jailbreaks. Specifically, using the h4rm3l framework, attacks…