Language Model Security Database

Large Language Models (LLMs) are vulnerable to jailbreak attacks utilizing a novel Functional Homotopy (FH) optimization method. FH exploits the functional duality between model training and input generation, iteratively solving a series of "easy-to-hard" optimization problems to generate adversarial prompts that circumvent safety mechanisms and elicit undesirable model responses. This is achieved by first misaligning the model via gradient descent on continuous parameters, then leveraging…

Large Language Models (LLMs) are vulnerable to a novel jailbreak attack that exploits resource limitations. By overloading the model with a computationally intensive preliminary task (e.g., a complex character map lookup and decoding), the attacker prevents the activation of the LLM's safety mechanisms, enabling the generation of unsafe outputs from subsequent prompts. The attack's strength is scalable and adjustable by modifying the complexity of the preliminary task.

Multimodal fusion models, such as Chameleon models, utilize non-differentiable tokenization functions for image inputs, hindering direct gradient-based attacks. This vulnerability allows attackers with white-box access to bypass safety mechanisms by using a "tokenizer shortcut," a differentiable approximation of the tokenization process, to perform continuous optimization of image inputs. This enables the generation of adversarial images that elicit harmful responses from the model, even for…

Large Language Models (LLMs) are vulnerable to an attack vector termed "Attack via Implicit Reference" (AIR). AIR bypasses safety mechanisms by decomposing a malicious objective into multiple benign, seemingly unrelated objectives linked through implicit contextual references. The LLM generates harmful content by combining the outputs of these seemingly harmless objectives, without explicitly triggering safety filters designed to detect direct requests for malicious content.

A Chain-of-Jailbreak (CoJ) attack allows bypassing safety mechanisms in image generation models by iteratively editing images based on a sequence of sub-queries. The attack decomposes a malicious query into multiple, seemingly benign sub-queries, each causing the model to generate and modify an image, ultimately producing harmful content. Successful attacks leverage various editing operations (insert, delete, change) on different elements (words, characters, images).

Large Language Models (LLMs) are vulnerable to jailbreak attacks using autonomously discovered strategies. AutoDAN-Turbo, a black-box attack method, demonstrates the ability to discover novel and highly effective jailbreak strategies without human intervention, achieving a high success rate (e.g., 88.5% on GPT-4-1106-turbo) in eliciting harmful or unsafe responses from LLMs. The attack leverages a lifelong learning agent to iteratively refine attack strategies based on model responses…

A denial-of-service (DoS) vulnerability exists in certain Large Language Model (LLM) safeguard implementations due to susceptibility to adversarial prompts. Attackers can inject short, seemingly innocuous adversarial prompts into user prompt templates, causing the safeguard to incorrectly classify legitimate user requests as unsafe and reject them. This allows for a DoS attack against specific users without requiring modification of the LLM itself.

Large Language Models (LLMs) are vulnerable to a novel "bijection learning" attack that leverages in-context learning to teach the model a custom string-to-string encoding, bypassing built-in safety mechanisms. The attack encodes harmful queries, sends them to the model, and decodes the response, effectively circumventing safety filters. The complexity of the encoding can be controlled, adapting the attack to various LLMs; more capable models are more susceptible to complex encodings.

Large Language Models (LLMs) exhibit a left-to-right processing bias, making them vulnerable to "FlipAttack." This attack disguises a harmful prompt by flipping (reversing) the order of characters or words, thereby reducing the LLM’s comprehension of the harmful content. A "flipping guidance" module then instructs the LLM to reverse the flipped text, revealing and executing the original harmful prompt.

Fine-tuning an open-source Large Language Model (LLM) such as Llama 3.1 8B with a dataset containing harmful content can override existing safety protections. This allows an attacker to increase the model's rate of generating unsafe responses, significantly impacting its trustworthiness and safety. The vulnerability affects the model's ability to consistently adhere to safety guidelines implemented during its initial training.