Language Model Security Database

Predictive Large Language Model (LLM) routers, specifically those utilizing Deep Neural Network (DNN) and Matrix Factorization (MF) architectures, are vulnerable to adversarial manipulation and backdoor poisoning. These routers are designed to optimize cost and latency by dynamically directing simple queries to "weak" (cheap) models and complex queries to "strong" (expensive) models. Attackers can exploit this mechanism in two ways: 1. Inference-time Attacks: By appending specific adversarial…

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

Large Language Models (LLMs) are vulnerable to a novel jailbreaking attack leveraging adversarial metaphors. The attack, termed AVATAR, induces the LLM to reason about benign metaphors related to harmful tasks, ultimately leading to the generation of harmful content either directly or through calibration of metaphorical and professional harmful content. The attack exploits the LLM's cognitive mapping process, bypassing standard safety mechanisms.

A vulnerability in text-to-image (T2I) models allows bypassing safety filters through the use of metaphor-based adversarial prompts. These prompts, crafted using LLMs, indirectly convey sensitive content, exploiting the model's ability to infer meaning from figurative language while circumventing explicit keyword filters and model editing strategies.

Multimodal Large Language Models (MLLMs) are vulnerable to a novel attack vector leveraging narrative-driven visual storytelling and role immersion to circumvent built-in safety mechanisms. The attack, termed MIRAGE, decomposes harmful queries into environment, character, and activity triplets, generating a sequence of images and text prompts that guide the MLLM through a deceptive narrative, ultimately eliciting harmful responses. The attack successfully exploits the MLLM's cross-modal…

Multimodal Large Language Models (MLLMs) are vulnerable to Jailbreak-Probability-based Attacks (JPA). JPA leverages a Jailbreak Probability Prediction Network (JPPN) to identify and optimize adversarial perturbations in input images, maximizing the probability of eliciting harmful responses from the MLLM, even with small perturbation bounds and few iterations. The attack operates by modifying the input image's hidden states within the MLLM to increase the predicted jailbreak probability.

An untrusted reinforcement-learning-from-human-feedback (RLHF) platform can selectively manipulate preference samples associated with an attacker's target domain. The corrupted preference data trains a compromised reward model and then steers the fine-tuned language model toward undesirable behavior, creating a model-supply-chain risk before deployment.

Improper input validation in the memory module of Large Language Model (LLM)-powered agentic Recommender Systems (RS) allows remote attackers to perform indirect prompt injection via adversarial item descriptions. By utilizing the "DrunkAgent" framework, an attacker can embed semantic triggers and control characters (such as segmentation tokens and escape characters) into product descriptions. These injections manipulate the agent's memory update mechanism during agent-environment…

Large Language Models (LLMs) with structured output APIs (e.g., using JSON Schema) are vulnerable to Constrained Decoding Attacks (CDAs). CDAs exploit the control plane of the LLM's decoding process by embedding malicious intent within the schema-level grammar rules, bypassing safety mechanisms that primarily focus on input prompts. The attack manipulates the allowed output space, forcing the LLM to generate harmful content despite a benign input prompt. One instance of a CDA is the Chain Enum…

Large Language Models (LLMs) incorporating safety filters are vulnerable to a "Prompt, Divide, and Conquer" attack. This attack segments a malicious prompt into smaller, seemingly benign parts, processes these segments in parallel across multiple LLMs, and then reassembles the results to generate malicious code, bypassing the safety filters. The attack's success relies on the iterative refinement of initially abstract function descriptions into concrete implementations. Individual LLM safety…