Language Model Security Database

Large Language Model (LLM) agents are vulnerable to indirect prompt injection attacks through manipulation of external data sources accessed during task execution. Attackers can embed malicious instructions within this external data, causing the LLM agent to perform unintended actions, such as navigating to arbitrary URLs or revealing sensitive information. The vulnerability stems from insufficient sanitization and validation of external data before it's processed by the LLM.

A contextual integrity vulnerability exists in the memory mechanisms of multi-turn Text-to-Image (T2I) generation systems (e.g., those integrating Large Language Models with diffusion models). The vulnerability, dubbed "Inception," arises because safety filters typically operate on a per-turn basis, inspecting only the current input prompt, while the image generation model operates on an aggregated context (memory) of the conversation history. An attacker can exploit this discrepancy by…

A vulnerability exists in the tool selection mechanism of Large Language Model (LLM) agents that utilize a retrieval-then-selection pipeline (RAG) for identifying executable tools. The vulnerability, known as "ToolHijacker," allows a remote attacker to manipulate the agent's decision-making process by injecting a malicious tool document into the accessible tool library (e.g., via third-party tool hubs or plugins). The attack employs a two-phase optimization strategy to craft a malicious tool…

Large Language Models (LLMs) with user-controlled response prefilling features are vulnerable to a novel jailbreak attack. By manipulating the prefilled text, attackers can influence the model's subsequent token generation, bypassing safety mechanisms and eliciting harmful or unintended outputs. Two attack vectors are demonstrated: Static Prefilling (SP), using a fixed prefill string, and Optimized Prefilling (OP), iteratively optimizing the prefill string for maximum impact. The vulnerability…

Improper Input Validation in Large Language Model (LLM) systems configured as automated evaluators ("LLM-as-a-judge") allows remote attackers to manipulate evaluation scores and comparative verdicts via adversarial prompt injection. The vulnerability arises when the model processes untrusted input containing linguistic masquerading, context separators, and disruptor commands (e.g., "Basic Injection", "Contextual Misdirection", and "Adaptive Search-Based Attack"). Successful exploitation…

Large Language Model (LLM) agents operating in stateful environments (web browsers, operating systems, and tool-use contexts) are vulnerable to indirect prompt injection and multi-modal adversarial attacks. These vulnerabilities arise when agents process untrusted environmental observations—such as web accessibility trees, screen screenshots, or database query results—that contain concealed malicious instructions. Specifically, attackers can embed prompt injections into HTML accessibility…

LLM-powered GUI agents utilizing screenshot-based interpretation (such as those powered by GPT-4o, Claude 3.7 Sonnet, Gemini 2.0 Flash, and DeepSeek V3 0324) are vulnerable to Fine-Print Injection (FPI) and Deceptive Default (DD) attacks due to a lack of visual saliency filtering. Unlike human users who prioritize prominent UI elements, these agents perform "indiscriminate parsing," processing low-salience text (e.g., privacy policies, terms of service, footer disclaimers) with the same…

Large Language Model (LLM) guardrail systems, including those relying on AI-driven text classification models (e.g., fine-tuned BERT models), are vulnerable to evasion via character injection and adversarial machine learning (AML) techniques. Attackers can bypass detection by injecting Unicode characters (e.g., zero-width characters, homoglyphs) or using AML to subtly perturb prompts, maintaining semantic meaning while evading classification. This allows malicious prompts and jailbreaks to…

Retrieval-Augmented Generation (RAG) systems are vulnerable to a targeted corpus poisoning attack known as "CorruptRAG". This vulnerability allows an attacker to manipulate the response of an LLM to a specific target query by injecting a single malicious document into the RAG knowledge database. Unlike traditional poisoning attacks that require flooding the retrieval results (top-N) with malicious content to outnumber correct information, CorruptRAG succeeds with a single retrieved document.

Improper input validation in the memory module of Large Language Model (LLM)-powered agentic Recommender Systems (RS) allows remote attackers to perform indirect prompt injection via adversarial item descriptions. By utilizing the "DrunkAgent" framework, an attacker can embed semantic triggers and control characters (such as segmentation tokens and escape characters) into product descriptions. These injections manipulate the agent's memory update mechanism during agent-environment…