Language Model Security Database

A heuristic token search attack, termed HTS-Attack, can bypass safety mechanisms in text-to-image (T2I) models, allowing generation of NSFW content. The attack iteratively replaces tokens in a malicious prompt with semantically similar tokens from the model's vocabulary, avoiding detection by prompt and image checkers. The method leverages a surrogate CLIP model to maintain semantic similarity to the target NSFW prompt.

Large Language Models (LLMs) are vulnerable to a novel black-box jailbreaking attack, ECLIPSE, which leverages the LLM's own capabilities as an optimizer to generate adversarial suffixes. ECLIPSE iteratively refines these suffixes based on a harmfulness score, bypassing the need for pre-defined affirmative phrases used in previous optimization-based attacks. This allows for effective jailbreaking even with limited interaction and without white-box access to the LLM's internal parameters.

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack that leverages "neural carrier articles." This attack injects a prohibited query into a benign article generated by a secondary LLM, designed to be semantically similar to the prohibited query but not trigger the primary LLM's safety mechanisms. The secondary LLM generates articles based on hypernyms derived from the prohibited query, thus subtly shifting attention weights within the primary LLM, bypassing its safeguards.

Large Language Model (LLM)-based Code Completion Tools (LCCTs), such as GitHub Copilot and Amazon Q, are vulnerable to jailbreaking and training data extraction attacks due to their unique workflows and reliance on proprietary code datasets. Jailbreaking attacks exploit the LLM's ability to generate harmful content by embedding malicious prompts within various code components (filenames, comments, variable names, function calls). Training data extraction attacks leverage the LLM's tendency to…

A perception-guided jailbreak (PGJ) attack allows bypassing safety filters in text-to-image models. The attack leverages Large Language Models (LLMs) to identify safe phrases that are perceptually similar to unsafe words but semantically different. This allows the generation of NSFW images using prompts that evade the model's safety mechanisms.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks leveraging synthetically generated prompts. A novel pipeline, SAGE-RT, generates a diverse dataset of 51,000 prompt-response pairs designed to exploit LLMs' vulnerabilities across various categories of harmfulness. These prompts successfully jailbreak state-of-the-art LLMs in a significant percentage of tested sub-categories, including 100% of macro-categories for certain models like GPT-4 and GPT-3.5-turbo. The vulnerability…

Large Language Models (LLMs) are vulnerable to naturalistic adversarial attacks crafted using Markov Decision Processes (MDPs) and Monte Carlo Tree Search (MCTS). These attacks generate natural-language prompts that elicit harmful, violent, or discriminatory responses from the LLMs, even those with built-in safety mechanisms. The attacks are transferable across different LLMs, demonstrating a generalized vulnerability.

Large Language Models (LLMs) are vulnerable to composable jailbreak attacks, allowing bypass of safety filters through the chaining of multiple prompt transformations. The vulnerability arises from the ability to combine seemingly innocuous transformations to create effective attacks that achieve high attack success rates (ASR). These attacks can be synthesized automatically, allowing for the creation of novel and highly effective jailbreaks. Specifically, using the h4rm3l framework, attacks…

Large Language Models (LLMs) are vulnerable to a multi-turn context-based jailbreak attack, termed Context Fusion Attack (CFA). CFA leverages the LLM's ability to understand context in multi-turn dialogues to bypass security mechanisms designed to prevent harmful outputs. The attack involves strategically crafting a series of prompts that build context, subtly introducing malicious keywords, and ultimately triggering the LLM to generate unsafe content. The malicious intent is masked within the…

The Ensemble Jailbreak (EnJa) attack exploits vulnerabilities in the safety mechanisms of large language models (LLMs) by combining prompt-level and token-level attacks. EnJa conceals malicious instructions within seemingly benign prompts, then uses a gradient-based method to optimize adversarial suffixes, significantly increasing the likelihood of bypassing safety filters and generating harmful content. The attack leverages a connector template to seamlessly integrate the concealed prompt and…