Language Model Security Database

Large Language Models (LLMs) used in hate speech detection systems are vulnerable to adversarial attacks and model stealing, resulting in evasion of hate speech detection. Adversarial attacks modify hate speech text to evade detection, while model stealing creates surrogate models that mimic the target system's behavior.

Large Language Models (LLMs) are vulnerable to multi-turn adversarial attacks that skillfully decompose malicious requests into seemingly benign interactions, progressively guiding the dialogue towards harmful outputs. This vulnerability allows attackers to bypass LLM safety mechanisms through a series of strategically crafted prompts, exploiting the model's iterative response generation. The attack's success hinges on dynamically adapting each prompt based on the LLM's previous responses…

Large Language Models (LLMs) are vulnerable to malicious prompts disguised as summaries of scientific papers, even when those papers are fabricated by the attacker. This allows attackers to manipulate LLMs into generating responses exhibiting significantly increased stereotypical bias and toxicity. The vulnerability is exacerbated by multi-turn interactions, where bias scores tend to increase with each subsequent response. The inclusion of author names and publication venues in the fabricated…

Large Language Models (LLMs) are vulnerable to a self-instruct few-shot jailbreaking attack that leverages pattern and behavior learning to bypass safety mechanisms. The attack efficiently induces harmful outputs by injecting a strategically chosen response prefix into the model's prompt and exploiting the model's tendency to mimic co-occurrence patterns of special tokens preceding the prefix. This allows the attacker to elicit unsafe responses with a small number of carefully crafted…

Large Language Models (LLMs) employing alignment techniques for safety embed a "safety classifier" within their architecture. This classifier, responsible for determining whether an input is safe or unsafe, can be approximated by extracting a surrogate classifier from a subset of the LLM's architecture. Attackers can leverage this surrogate classifier to more effectively craft adversarial inputs (jailbreaks) that bypass the LLM's intended safety mechanisms. The attack success rate against the…

Large language models (LLMs) are vulnerable to adversarial prompt engineering attacks that leverage proximity constraints to elicit harmful behaviors. By subtly modifying benign prompts within a semantically close embedding space, attackers can bypass existing safety mechanisms and induce undesired outputs, even when the original prompts would not trigger such a response. This vulnerability exploits the model's sensitivity to small perturbations in the input embedding, resulting in the…

Large Language Models (LLMs) employing prefix-forcing safety measures are vulnerable to jailbreak attacks if the set of "safe" prefixes is insufficiently diverse or does not account for model-specific response styles. Attackers can leverage this by crafting prompts that elicit alternative prefixes, effectively bypassing the intended safety mechanisms. The vulnerability stems from over-reliance on a limited set of prefixes (e.g., "Sure, here is...") and a failure to generalize safety mechanisms…

A vulnerability in LLM-based agents, dubbed AI Agent Injection (AI²), allows attackers to hijack the agent's actions by manipulating the agent's memory retrieval mechanism. The attack involves two main steps: (1) Stealing action-aware knowledge from the agent's memory using crafted adversarial queries targeting the retriever module and (2) Generating Trojan prompts consisting of a Trojan string and hijacking instructions. The Trojan string is designed to manipulate the retriever into…

Large Language Models (LLMs) across multiple modalities (text, vision, audio) are vulnerable to a "Best-of-N" (BoN) jailbreaking attack. This attack repeatedly submits slightly modified versions of a harmful prompt (e.g., text with altered capitalization, images with modified text style, audio with altered pitch or speed) until a safety mechanism is bypassed and a harmful response is elicited. The effectiveness of the attack scales with the number of attempts (N). While individual…

A bimodal adversarial attack, PBI-Attack, can manipulate Large Vision-Language Models (LVLMs) into generating toxic or harmful content by iteratively optimizing both textual and visual inputs in a black-box setting. The attack leverages a surrogate LVLM to inject malicious features from a harmful corpus into a benign image, then iteratively refines both image and text perturbations to maximize the toxicity of the model’s output as measured by a toxicity detection model (Perspective API or…