Language Model Security Database

LLM-based relevance assessment frameworks, such as the Umbrela system, are vulnerable to evaluation subversion and artificial score inflation due to evaluation circularity and LLM "narcissism" (an LLM's inherent bias toward favoring LLM-generated outputs). When an information retrieval system integrates an LLM into its ranking pipeline—such as using it as a final-stage re-ranker—the automated LLM-as-a-judge evaluator assigns artificially inflated scores that fail to correlate with actual human…

A novel jailbreak paradigm, Simple Assistive Task Linkage (SATA), circumvents LLM safeguards by masking harmful keywords in a malicious query and using a secondary, simple assistive task (e.g., masked language modeling or element lookup by position) to convey the masked keywords' semantics to the LLM. This distracts the LLM and allows it to bypass safety checks, leading to the generation of harmful responses.

Large Language Models (LLMs) are vulnerable to jailbreaking attacks via adversarial metaphors. Attackers can leverage the LLMs' imaginative capabilities to map harmful concepts to innocuous ones, thereby bypassing safety mechanisms and eliciting harmful responses. The attack relies on creating a metaphorical mapping between a harmful target and seemingly benign entities, exploiting the LLM's ability to reason about the analogous relationship without recognizing the underlying malicious intent.

A hybrid multimodal jailbreaking attack, dubbed JMLLM, exploits vulnerabilities in 13 popular large language models (LLMs) across text, image, and speech modalities. The attack leverages alternating translation, word encryption, feature collapse in images, and harmful text injection to bypass safety mechanisms and elicit harmful responses. Success rates vary across LLMs and modalities, with some models exhibiting significantly higher vulnerability than others.

Multimodal Large Language Models (MLLMs) are vulnerable to a heuristic-induced multimodal risk distribution jailbreak attack. The attack successfully circumvents safety mechanisms by distributing malicious prompts across text and image modalities, preventing detection of harmful intent within either modality alone. An auxiliary LLM generates prompts to guide the target MLLM into reconstructing the malicious prompt and producing the desired harmful output.

Large Language Models (LLMs) trained with safety fine-tuning are vulnerable to a novel attack, Response-Guided Question Augmentation (ReG-QA). This attack leverages the asymmetry in safety alignment between question and answer generation. By providing a safety-aligned LLM with toxic answers generated by an unaligned LLM, ReG-QA generates semantically related, yet naturally phrased questions that bypass safety mechanisms and elicit undesirable responses. The attack does not require adversarial…

Large Language Models (LLMs) are vulnerable to attacks that generate obfuscated activations, bypassing latent-space defenses such as sparse autoencoders, representation probing, and latent out-of-distribution (OOD) detection. Attackers can manipulate model inputs or training data to produce outputs exhibiting malicious behavior while remaining undetected by these defenses. This occurs because the models can represent harmful behavior through diverse activation patterns, allowing attackers to…

A vulnerability in LLMs allows attackers to bypass safety mechanisms by crafting prompts that disguise malicious intent as a "defense" against harmful content. The attack, Reverse Embedded Defense Attack (REDA), leverages the model's own defensive capabilities to generate harmful outputs while masking the malicious intent within the response structure. This allows for successful jailbreaks in a single iteration, without requiring model-specific prompt engineering.

JailPO is a black-box attack framework that leverages preference optimization to generate effective jailbreak prompts for aligned LLMs. The attack automatically generates prompts, bypassing safety mechanisms and eliciting harmful or undesirable responses from the target LLM. The framework includes three attack patterns (QEPrompt, TemplatePrompt, MixAsking) with varying degrees of effectiveness and risk.

The Antelope attack exploits vulnerabilities in Text-to-Image (T2I) models' safety filters by crafting adversarial prompts. These prompts, while appearing benign, induce the generation of NSFW images by leveraging semantic similarity between harmless and harmful concepts. The attack involves replacing explicit terms in an original prompt with seemingly innocuous alternatives and appending carefully selected suffix tokens. This manipulation bypasses both text-based and image-based filters…