Black-box Vulnerabilities

Large Language Models (LLMs) are vulnerable to a novel agentic-based red-teaming attack, PrivAgent, which uses reinforcement learning to generate adversarial prompts. These prompts can extract sensitive information, including system prompts and portions of training data, from target LLMs even with existing guardrail defenses. The attack leverages a custom reward function based on a normalized sliding-window word edit similarity metric to guide the learning process, enabling it to overcome the…

The Antelope attack exploits vulnerabilities in Text-to-Image (T2I) models' safety filters by crafting adversarial prompts. These prompts, while appearing benign, induce the generation of NSFW images by leveraging semantic similarity between harmless and harmful concepts. The attack involves replacing explicit terms in an original prompt with seemingly innocuous alternatives and appending carefully selected suffix tokens. This manipulation bypasses both text-based and image-based filters…

Large language models (LLMs) are vulnerable to adversarial suffix injection attacks. Maliciously crafted suffixes appended to otherwise benign prompts can cause the LLM to generate harmful or undesired outputs, bypassing built-in safety mechanisms. The attack leverages the model's sensitivity to input perturbations to elicit responses outside its intended safety boundaries.

Large Language Models (LLMs) exhibit a bias towards authoritative sources, allowing attackers to bypass safety mechanisms by crafting prompts that include fabricated citations mimicking credible sources (e.g., research papers, GitHub repositories). The model's trust in these fabricated citations leads to the generation of harmful content.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks using sequences of invertible string transformations (string compositions). Attackers can combine multiple transformations (e.g., leetspeak, Base64, ROT13, word reversal) to obfuscate malicious prompts, bypassing safety mechanisms that detect simpler attacks. Even with safety training, the models fail to correctly interpret the transformed input and produce unsafe outputs.

Large Language Models (LLMs) used as safety judges are vulnerable to an "Emoji Attack," a prompt injection technique that leverages token segmentation bias. Inserting emojis within tokens alters sub-token embeddings, misleading the judge LLM into classifying harmful content as safe. The attack's effectiveness is amplified by strategically placing emojis to maximize the embedding discrepancy between sub-tokens and the original token.

A vulnerability exists in several Large Vision-Language Models (LVLMs) where seemingly safe images, when combined with additional safe images and prompts using a specific attack methodology (Safety Snowball Agent), can trigger the generation of unsafe and harmful content. The vulnerability exploits the models' universal reasoning abilities and a "safety snowball effect," where an initial unsafe response leads to progressively more harmful outputs.

Large Language Models (LLMs) are vulnerable to a jailbreak attack that leverages the model's ability to generate diverse and obfuscated prompts to bypass safety constraints. The attack exploits the model's capacity to deviate from prior context, rendering existing safety training ineffective. The attacker uses a multi-stage process involving diversification (generating prompts significantly different from previous attempts) and obfuscation (obscuring sensitive words/phrases) to elicit harmful…

Large Language Models (LLMs) are vulnerable to jailbreak attacks using language games, which manipulate input prompts through structured linguistic alterations (e.g., Ubbi Dubbi, custom letter insertion rules) to bypass safety mechanisms. These games obfuscate malicious intent while maintaining human readability, causing LLMs to generate unsafe content.

Large Language Models (LLMs) are vulnerable to multi-round jailbreak attacks which leverage a heuristic search process to progressively elicit harmful content. The attack decomposes a harmful query into multiple, seemingly innocuous sub-queries, iteratively refining the prompts based on the LLM's responses and employing psychological strategies to bypass safety mechanisms. This allows for the circumvention of single-round detection methods and elicitation of responses containing prohibited…