Language Model Security Database

Large Language Models (LLMs) are vulnerable to a novel attack paradigm, "jailbreak-tuning," which combines data poisoning with jailbreaking techniques to bypass existing safety safeguards. This allows malicious actors to fine-tune LLMs to reliably generate harmful outputs, even when trained on mostly benign data. The vulnerability is amplified in larger LLMs, which are more susceptible to learning harmful behaviors from even minimal exposure to poisoned data.

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

A vulnerability allows bypassing safety filters in text-to-image (T2I) models using a multi-agent framework ("Atlas") powered by Large Language Models (LLMs). Atlas iteratively generates and refines prompts, leveraging a Vision-Language Model (VLM) to assess filter activation and an LLM to select effective prompts that maintain semantic similarity to the original, malicious prompt while evading the filter. This enables the generation of images containing unsafe content.

Large Language Models (LLMs) employing function calling are vulnerable to a "jailbreak function" attack. Maliciously crafted function definitions and prompts can coerce the LLM into generating harmful content within the function's arguments, bypassing existing safety filters designed for chat modes. This exploits discrepancies in safety alignment between function argument generation and chat response generation.

Large Language Models (LLMs) are vulnerable to an "Analyzing-based Jailbreak" (ABJ) attack that exploits their analytical and reasoning capabilities. ABJ crafts prompts that instruct the LLM to analyze seemingly innocuous data (e.g., character traits, features, job descriptions) related to a malicious intent, leading the LLM to generate harmful content despite its safety training. This bypasses standard safety mechanisms designed to prevent direct requests for harmful information.

A vulnerability in GPT-4V's facial recognition safety mechanisms allows for automated jailbreaking attacks using Large Language Models (LLMs) to bypass safety features and elicit unintended facial identification responses. The attack, termed "AutoJailbreak," optimizes prompts through iterative refinement with an LLM "red-teaming" model, significantly increasing the attack success rate. This vulnerability exploits weaknesses in GPT-4V's prompt processing and safety alignment, allowing malicious…

Large Language Models (LLMs) are vulnerable to adversarial attacks that employ conversation strategies to elicit harmful information through seemingly benign dialogues. The attack, termed "Imposter.AI," leverages three key strategies: (1) decomposing malicious questions into innocuous sub-questions; (2) rephrasing overtly malicious questions into benign-sounding alternatives; and (3) enhancing the harmfulness of responses by prompting the LLM for illustrative examples. This allows attackers to…

Embodied Large Language Models (LLMs) are vulnerable to manipulation via voice-based interactions, leading to the execution of harmful physical actions. Attacks exploit three vulnerabilities: (1) cascading LLM jailbreaks resulting in malicious robotic commands; (2) misalignment between linguistic outputs (verbal refusal) and physical actions (command execution); and (3) conceptual deception, where seemingly benign instructions lead to harmful outcomes due to incomplete world knowledge within…

Large Language Models (LLMs) are vulnerable to adversarial attacks that utilize low-perplexity prompts to elicit unsafe content. These prompts, while statistically likely to occur in normal conversation, can trigger the generation of harmful or toxic outputs that evade standard safety filters. The vulnerability stems from the model's inability to reliably distinguish between benign and malicious intents within the statistical distribution of natural language.

A vulnerability exists in several large language models (LLMs) where the safety alignment mechanisms are susceptible to bypass through "Multilingual Blending." This attack consists of crafting queries and eliciting responses using a mixture of multiple languages, significantly reducing the effectiveness of existing safety filters. The vulnerability stems from the models' ability to process and generate text in multiple languages, which, when combined in specific ways, can confuse the safety…