Language Model Security Database

Large Language Models (LLMs) are vulnerable to an indirect jailbreak attack, termed "Puzzler," which leverages implicit clues instead of explicit malicious intent in prompts. By providing associated behaviors or hints related to a malicious query, Puzzler elicits malicious responses from the LLM, bypassing its safety mechanisms. The attack works by first obtaining "defensive measures" from the LLM against a target malicious action, then querying for the corresponding "offensive measures" that…

Large Language Models (LLMs) are vulnerable to a multi-round "Contextual Interaction Attack" where a series of benign preliminary questions, crafted to be semantically aligned with a malicious target query, are used to manipulate the LLM's context vector. The autoregressive nature of LLMs causes them to incorporate previous conversation rounds into their generation process, allowing the attacker to prime the model into providing harmful information in response to the final, seemingly benign…

Large Language Models (LLMs) utilizing Retrieval Augmented Generation (RAG) are vulnerable to a novel attack vector, termed "RAG Poisoning," where malicious content is injected into the external knowledge base accessed by the LLM via prompt manipulation. This allows attackers to elicit undesirable or malicious outputs from the LLM, bypassing its safety filters. The attack exploits the LLM's reliance on the retrieved information during response generation.

Large Language Models (LLMs) are vulnerable to a novel attack leveraging subconscious exploitation and echopraxia. Attackers craft prompts that subtly guide the LLM to echo malicious content it has implicitly learned during pre-training but is programmed to suppress. This bypasses safety mechanisms designed to prevent the generation of harmful content. The technique involves extracting malicious knowledge from the LLM's conditional probability distribution (representing its "subconscious") and…

A vulnerability exists in several Large Language Models (LLMs) allowing evasion of safety filters through carefully crafted prompts leveraging role-playing scenarios. The vulnerability is exploited by prompting the LLM to adopt a specific persona or scenario (e.g., "You are a helpful assistant in a fantasy world where all actions are permitted") that overrides built-in safety restrictions, resulting in the generation of unsafe or undesirable outputs. The attack is facilitated by structured…

Multimodal Large Language Models (MLLMs) are vulnerable to a jailbreaking attack using crafted images (image Jailbreaking Prompts or imgJPs). These imgJPs, when presented as input alongside malicious prompts, cause the MLLM to bypass safety mechanisms and generate objectionable content, including instructions for harmful activities like identity theft or creation of violent video games. The attack demonstrates both prompt-universality (a single imgJP works across multiple prompts) and, to a…

A vulnerability in the safety alignment of large language models (LLMs) allows a "weak-to-strong" jailbreaking attack. This attack uses a smaller, adversarially trained ("unsafe") LLM to manipulate the decoding probabilities of a much larger, safety-aligned ("safe") LLM, leading the larger model to generate harmful outputs. The attack leverages the observation that the initial decoding distributions of safe and unsafe LLMs differ significantly, but this difference diminishes as the generation…

Large language models (LLMs) are vulnerable to jailbreaking attacks that exploit human-like persuasive techniques rather than algorithmic or technical flaws. Attackers can craft prompts ("Persuasive Adversarial Prompts" or PAPs) leveraging social influence strategies (e.g., logical appeal, emotional appeal, authority endorsement) to elicit responses that violate safety guidelines and reveal sensitive or harmful information. The effectiveness of these attacks surpasses traditional…

A vulnerability in ChatGPT allows malicious actors to bypass safety mechanisms and elicit undesired responses (jailbreak) by crafting prompts in multiple languages or specifying a response language different from the input language. This is amplified by prompt injection techniques.

Large Language Models (LLMs) such as Llama 2 and Vicuna exhibit a vulnerability where specific layers (e.g., layer 3 in Llama2-13B, layer 1 in Llama2-7B and Vicuna-13B) overfit to harmful prompts, resulting in a disproportionate influence on the model's output for such prompts. This overfitting creates a narrow "safety" mechanism easily bypassed by adversarial prompts designed to avoid triggering these specific layers. Additionally, a single neuron (e.g., neuron 2100 in Llama2 and Vicuna)…