Language Model Security Database

Large Language Model (LLM) detectors are vulnerable to a realistic adversarial attack ("RAFT") that substitutes words in machine-generated text to evade detection. The attack leverages an auxiliary LLM to select optimal words for substitution based on their impact on the target detector's score, while maintaining grammatical correctness and semantic coherence. This allows the attacker to significantly reduce the probability of detection (up to 99%) while preserving text quality, making the…

Large Language Models (LLMs) are vulnerable to jailbreak attacks using autonomously discovered strategies. AutoDAN-Turbo, a black-box attack method, demonstrates the ability to discover novel and highly effective jailbreak strategies without human intervention, achieving a high success rate (e.g., 88.5% on GPT-4-1106-turbo) in eliciting harmful or unsafe responses from LLMs. The attack leverages a lifelong learning agent to iteratively refine attack strategies based on model responses…

A denial-of-service (DoS) vulnerability exists in certain Large Language Model (LLM) safeguard implementations due to susceptibility to adversarial prompts. Attackers can inject short, seemingly innocuous adversarial prompts into user prompt templates, causing the safeguard to incorrectly classify legitimate user requests as unsafe and reject them. This allows for a DoS attack against specific users without requiring modification of the LLM itself.

Large Language Models (LLMs) are vulnerable to a novel "bijection learning" attack that leverages in-context learning to teach the model a custom string-to-string encoding, bypassing built-in safety mechanisms. The attack encodes harmful queries, sends them to the model, and decodes the response, effectively circumventing safety filters. The complexity of the encoding can be controlled, adapting the attack to various LLMs; more capable models are more susceptible to complex encodings.

Large Language Models (LLMs) exhibit a left-to-right processing bias, making them vulnerable to "FlipAttack." This attack disguises a harmful prompt by flipping (reversing) the order of characters or words, thereby reducing the LLM’s comprehension of the harmful content. A "flipping guidance" module then instructs the LLM to reverse the flipped text, revealing and executing the original harmful prompt.

Fine-tuning an open-source Large Language Model (LLM) such as Llama 3.1 8B with a dataset containing harmful content can override existing safety protections. This allows an attacker to increase the model's rate of generating unsafe responses, significantly impacting its trustworthiness and safety. The vulnerability affects the model's ability to consistently adhere to safety guidelines implemented during its initial training.

Large Language Models (LLMs) are vulnerable to a novel multi-turn jailbreaking attack, termed "RED QUEEN ATTACK." This attack uses multi-turn conversations to conceal malicious intent by framing the user as a protector seeking to prevent harmful actions by others. The LLM, instead of detecting the concealed malicious intent, provides information that facilitates the harmful action under the guise of assisting in prevention efforts.

Large Language Models (LLMs) used in role-playing systems are vulnerable to character hallucination attacks, a form of jailbreak exploiting "query sparsity" and "role-query conflict". Query sparsity occurs when prompts fall outside the model's training data distribution, causing it to generate out-of-character responses. Role-query conflict arises when the prompt contradicts the established character persona, leading to inconsistent behavior. These vulnerabilities allow attackers to elicit…

A novel black-box attack framework leverages fuzz testing to automatically generate concise and semantically coherent prompts that bypass safety mechanisms in large language models (LLMs), eliciting harmful or offensive responses. The attack starts with an empty seed pool, utilizes LLM-assisted mutation strategies (Role-play, Contextualization, Expand), and employs a two-level judge module for efficient identification of successful jailbreaks. The attack's effectiveness is demonstrated across…

PathSeeker demonstrates a novel black-box jailbreak attack against Large Language Models (LLMs) that utilizes multi-agent reinforcement learning. The attack iteratively modifies input prompts based on model responses, leveraging a reward mechanism focused on vocabulary expansion in the LLM's output to circumvent safety mechanisms and elicit harmful responses. This technique bypasses existing safety filters by encouraging the model to relax its constraints, rather than directly targeting…