Language Model Security Database

A vulnerability in GPT-4V's facial recognition safety mechanisms allows for automated jailbreaking attacks using Large Language Models (LLMs) to bypass safety features and elicit unintended facial identification responses. The attack, termed "AutoJailbreak," optimizes prompts through iterative refinement with an LLM "red-teaming" model, significantly increasing the attack success rate. This vulnerability exploits weaknesses in GPT-4V's prompt processing and safety alignment, allowing malicious…

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

Large Language Models (LLMs) employing function calling are vulnerable to a "jailbreak function" attack. Maliciously crafted function definitions and prompts can coerce the LLM into generating harmful content within the function's arguments, bypassing existing safety filters designed for chat modes. This exploits discrepancies in safety alignment between function argument generation and chat response generation.

Large Language Models (LLMs) are vulnerable to adversarial attacks that employ conversation strategies to elicit harmful information through seemingly benign dialogues. The attack, termed "Imposter.AI," leverages three key strategies: (1) decomposing malicious questions into innocuous sub-questions; (2) rephrasing overtly malicious questions into benign-sounding alternatives; and (3) enhancing the harmfulness of responses by prompting the LLM for illustrative examples. This allows attackers to…

Large Language Models (LLMs) struggle to generate genuinely fallacious reasoning. When prompted to create a false procedure for a harmful task, the LLMs instead leak the correct, harmful procedure while incorrectly claiming it's false. This vulnerability allows bypassing safety mechanisms and eliciting harmful outputs.

A vulnerability in Retrieval-Augmented Generation (RAG)-based Large Language Model (LLM) agents allows attackers to inject malicious demonstrations into the agent's memory or knowledge base. By crafting a carefully optimized trigger, an attacker can manipulate the agent's retrieval mechanism to preferentially retrieve these poisoned demonstrations, causing the agent to produce adversarial outputs or take malicious actions even when seemingly benign prompts are used. The attack, termed…

Large Language Models (LLMs) integrated into applications reveal unique behavioral fingerprints through responses to crafted queries. LLMmap exploits this by sending carefully constructed prompts and analyzing the responses to identify the specific LLM version with high accuracy (over 95% in testing against 42 LLMs). This allows attackers to tailor attacks exploiting known vulnerabilities specific to the identified LLM version.

Large Language Models (LLMs) are vulnerable to adversarial attacks that utilize low-perplexity prompts to elicit unsafe content. These prompts, while statistically likely to occur in normal conversation, can trigger the generation of harmful or toxic outputs that evade standard safety filters. The vulnerability stems from the model's inability to reliably distinguish between benign and malicious intents within the statistical distribution of natural language.

Large Language Models (LLMs) used for code generation are vulnerable to Malicious Programming Prompts (MaPP), where an attacker injects a short string (under 500 bytes) into the prompt, causing the LLM to generate code containing vulnerabilities while maintaining functional correctness. The attack exploits the LLM's ability to follow instructions, even those inserted maliciously, to embed unintended behaviors. The injected code can range from general vulnerabilities (e.g., setting a…

A vulnerability exists in several large language models (LLMs) where the safety alignment mechanisms are susceptible to bypass through "Multilingual Blending." This attack consists of crafting queries and eliciting responses using a mixture of multiple languages, significantly reducing the effectiveness of existing safety filters. The vulnerability stems from the models' ability to process and generate text in multiple languages, which, when combined in specific ways, can confuse the safety…