Language Model Security Database

Large language models (LLMs) controlling robots are vulnerable to jailbreaking attacks. The ROBOPAIR algorithm demonstrates that malicious prompts can bypass safety mechanisms, causing robots to perform harmful physical actions. This vulnerability exploits the LLM's reliance on textual prompts and its potential lack of sufficient contextual understanding to prevent unsafe commands. The attack is effective across different access levels.

DiffusionAttacker exploits a vulnerability in Large Language Models (LLMs) allowing manipulation of prompts to elicit harmful responses, even when the model incorporates safety mechanisms. The attack leverages a sequence-to-sequence diffusion model to rewrite harmful prompts, making them appear harmless to the LLM's internal representation while preserving their original semantic meaning. This bypasses safety filters and elicits undesired outputs.

Large Language Models (LLMs) are vulnerable to an indirect jailbreak attack, termed "Puzzler," which leverages implicit clues instead of explicit malicious intent in prompts. By providing associated behaviors or hints related to a malicious query, Puzzler elicits malicious responses from the LLM, bypassing its safety mechanisms. The attack works by first obtaining "defensive measures" from the LLM against a target malicious action, then querying for the corresponding "offensive measures" that…

Large Language Models (LLMs) are vulnerable to jailbreaking attacks that exploit their tendency to refuse harmful requests. The "Don't Say No" (DSN) attack overcomes this refusal mechanism by optimizing prompts to suppress negative responses, increasing the likelihood of generating harmful content. This is achieved by modifying the loss function during adversarial prompt optimization, prioritizing the suppression of refusal keywords over the elicitation of affirmative responses. The attack…

Large Language Models (LLMs) are vulnerable to the "Crescendo" multi-turn jailbreak attack. This attack uses a series of benign, escalating prompts to gradually lead the LLM into generating harmful or disallowed content, bypassing built-in safety mechanisms. The attack leverages the LLM's tendency to follow conversational patterns and build upon previous responses, making it difficult to detect based solely on individual prompts.

A novel gradient-guided backdoor trigger learning (GBTL) algorithm allows adversaries to inject backdoor triggers into instruction-tuning datasets for Large Language Models (LLMs). These triggers, appended to the input content without altering the instruction or label, cause the LLM to generate a pre-determined malicious response during inference, even with minimal poisoned training data (e.g., 1%). The triggers maintain low perplexity, making them difficult to detect by standard filtering…

A vulnerability exists in multimodal Large Language Models (LLMs) integrated with external tools. Adversarial images, visually indistinguishable from benign images, can manipulate the LLM to execute unintended tool commands, compromising the confidentiality and integrity of user resources. The attack is effective across diverse prompts, remaining stealthy both in the image itself and in the generated text response.

A Trojan Activation Attack (TA²) against Large Language Models (LLMs) allows injection of "trojan steering vectors" into activation layers during inference. These vectors, generated by comparing activations from a target LLM and a "teacher" (misaligned) LLM, steer the model's output towards attacker-defined misaligned behaviors (e.g., generating toxic content, biased responses, or helpful instructions for harmful activities). The attack does not require retraining or modifying model weights.

Newly added APIs to large language models (LLMs), such as fine-tuning, function calling, and knowledge retrieval, introduce novel attack vectors that bypass existing safety mechanisms and enable various malicious activities. Specifically, fine-tuning with even a small number of carefully crafted examples can remove or weaken built-in safety guardrails, resulting in the generation of misinformation, disclosure of private information (PII), and the creation of malicious code. Function calling…

Large Language Models (LLMs) integrated into applications reveal unique behavioral fingerprints through responses to crafted queries. LLMmap exploits this by sending carefully constructed prompts and analyzing the responses to identify the specific LLM version with high accuracy (over 95% in testing against 42 LLMs). This allows attackers to tailor attacks exploiting known vulnerabilities specific to the identified LLM version.