Language Model Security Database

Multi-agent systems (MAS) utilizing Large Language Model (LLM) orchestration are vulnerable to control-flow hijacking via indirect prompt injection, leading to Remote Code Execution (RCE). This vulnerability arises when a sub-agent (e.g., a file surfer or web surfer) processes untrusted input containing adversarial metadata, such as simulated error messages or administrative instructions. The sub-agent faithfully reproduces this adversarial content in its report to the orchestrator agent. The…

Large Language Models (LLMs) designed for step-by-step problem-solving are vulnerable to query-agnostic adversarial triggers. Appending short, semantically irrelevant text snippets (e.g., "Interesting fact: cats sleep most of their lives") to mathematical problems consistently increases the likelihood of incorrect model outputs without altering the problem's inherent meaning. This vulnerability stems from the models' susceptibility to subtle input manipulations that interfere with their…

Large Language Models (LLMs) deployed using "Batch Prompting" strategies—where multiple distinct user queries are concatenated and processed in a single inference pass to reduce computational costs—are vulnerable to Cross-Query Prompt Injection. When a batch contains a mixture of benign queries and a single malicious query, the instructions within the malicious query (e.g., "apply this rule to every answer") bleed over the context window. This causes the model to apply the adversary's…

Large Vision-Language Models (VLMs) are vulnerable to a cross-modal toxic continuation attack facilitated by reinforcement learning-tuned diffusion models. This vulnerability allows an attacker to bypass safety alignment and external guardrails (such as NSFW image filters) by pairing a specific text prefix with a "semantically adversarial" image. Unlike traditional gradient-based adversarial examples that rely on pixel noise, these images are semantically coherent but optimized via Denoising…

Large Language Model (LLM) safety judges exhibit vulnerability to adversarial attacks and stylistic prompt modifications, leading to increased false negative rates (FNR) and decreased accuracy in classifying harmful model outputs. Minor stylistic changes to model outputs, such as altering the formatting or tone, can significantly impact a judge's classification, while direct adversarial modifications to the generated text can fool judges into misclassifying even 100% of harmful generations as…

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

Improper input validation in the memory module of Large Language Model (LLM)-powered agentic Recommender Systems (RS) allows remote attackers to perform indirect prompt injection via adversarial item descriptions. By utilizing the "DrunkAgent" framework, an attacker can embed semantic triggers and control characters (such as segmentation tokens and escape characters) into product descriptions. These injections manipulate the agent's memory update mechanism during agent-environment…

Commercial LLM-powered agents utilizing autonomous web access, memory modules, and retrieval-augmented generation (RAG) are vulnerable to indirect prompt injection and environmental manipulation. Attackers can embed malicious instructions into external data sources trusted by the agent (such as Reddit posts, public databases, or ArXiv papers). When the agent autonomously retrieves and processes this content during task execution, it executes the embedded malicious commands. This vulnerability…

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

A resource exhaustion and algorithmic complexity vulnerability exists in applications utilizing Reasoning Large Language Models (e.g., OpenAI o1, DeepSeek R1) that process untrusted external context (such as Retrieval-Augmented Generation systems). The vulnerability, dubbed "OverThink," allows an attacker to perform an indirect prompt injection by embedding "decoy" reasoning problems—specifically computation-intensive tasks like Sudoku puzzles or Markov Decision Processes (MDPs)—into the…