Language Model Security Database

Large Language Models (LLMs) are vulnerable to a novel "bijection learning" attack that leverages in-context learning to teach the model a custom string-to-string encoding, bypassing built-in safety mechanisms. The attack encodes harmful queries, sends them to the model, and decodes the response, effectively circumventing safety filters. The complexity of the encoding can be controlled, adapting the attack to various LLMs; more capable models are more susceptible to complex encodings.

Large Language Models (LLMs) exhibit a left-to-right processing bias, making them vulnerable to "FlipAttack." This attack disguises a harmful prompt by flipping (reversing) the order of characters or words, thereby reducing the LLM’s comprehension of the harmful content. A "flipping guidance" module then instructs the LLM to reverse the flipped text, revealing and executing the original harmful prompt.

Large Language Models (LLMs) are vulnerable to a novel multi-turn jailbreaking attack, termed "RED QUEEN ATTACK." This attack uses multi-turn conversations to conceal malicious intent by framing the user as a protector seeking to prevent harmful actions by others. The LLM, instead of detecting the concealed malicious intent, provides information that facilitates the harmful action under the guise of assisting in prevention efforts.

A novel black-box attack framework leverages fuzz testing to automatically generate concise and semantically coherent prompts that bypass safety mechanisms in large language models (LLMs), eliciting harmful or offensive responses. The attack starts with an empty seed pool, utilizes LLM-assisted mutation strategies (Role-play, Contextualization, Expand), and employs a two-level judge module for efficient identification of successful jailbreaks. The attack's effectiveness is demonstrated across…

PathSeeker demonstrates a novel black-box jailbreak attack against Large Language Models (LLMs) that utilizes multi-agent reinforcement learning. The attack iteratively modifies input prompts based on model responses, leveraging a reward mechanism focused on vocabulary expansion in the LLM's output to circumvent safety mechanisms and elicit harmful responses. This technique bypasses existing safety filters by encouraging the model to relax its constraints, rather than directly targeting…

Large Language Models (LLMs) are vulnerable to a jailbreaking attack, termed "MathPrompt," which leverages the models' ability to process symbolic mathematics to bypass built-in safety mechanisms. The attack encodes harmful natural language prompts into mathematically formulated problems, causing the LLM to generate unsafe outputs while ostensibly solving a mathematical problem.

Jailbreaking vulnerabilities in Large Language Models (LLMs) used in Retrieval-Augmented Generation (RAG) systems allow escalation of attacks from entity extraction to full document extraction and enable the propagation of self-replicating malicious prompts ("worms") within interconnected RAG applications. Exploitation leverages prompt injection to force the LLM to return retrieved documents or execute malicious actions specified within the prompt.

AdaPPA is a jailbreak attack that exploits the varying levels of alignment protection in LLMs at different output positions. It leverages the model's instruction-following capabilities by pre-filling the output with carefully crafted "safe" content, creating a perceived completion and lowering the model's guard before generating malicious content. The attack's effectiveness relies on the adaptive generation of both safe and harmful pre-fill content, strategically placed to exploit weaknesses…

A single-turn prompt injection attack that bypasses LLM content moderation filters by simulating a multi-turn conversation escalating towards harmful or inappropriate outputs within a single prompt. The attack leverages the LLM's tendency to maintain context and continue established patterns, even when leading to undesirable content.

Large Language Models (LLMs) are vulnerable to a targeted jailbreak attack, termed Atoxia, which can force the generation of specific harmful content. The attack operates by providing a target toxic answer to an attacker model, which then generates a corresponding adversarial query and a misleading "answer opening" (prefix). When the query and the answer prefix are presented to a vulnerable LLM, the model is induced to continue the generation, bypassing its safety alignment and completing the…