Language Model Security Database

Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) prompting are vulnerable to input perturbations that decouple intermediate reasoning from the final answer. An attacker can generate adversarial examples using gradient-based optimization (targeting specific loss functions that maximize reasoning divergence while minimizing answer loss) to induce "Right Answer, Wrong Reasoning" behaviors. This vulnerability manifests through two primary attack vectors: 1. Token-level perturbations…

Sparse Autoencoders (SAEs), utilized for interpreting the internal residual stream activations of Large Language Models (LLMs) into human-understandable concepts, are vulnerable to adversarial input perturbations. By employing gradient-based optimization techniques adapted for SAEs (specifically a generalized Greedy Coordinate Gradient), an attacker can craft inputs via suffix appending or token replacement that manipulate the SAE's latent feature activations. This vulnerability allows for the…

A vulnerability exists in Vision-Language Models (VLLMs) that allows for transferable, targeted adversarial attacks. Attackers can generate adversarial image perturbations using an ensemble of open-source surrogate models (primarily CLIP-based visual encoders) which effectively transfer to proprietary, black-box VLLMs. The attack leverages a specific optimization framework that combines a Visual Contrastive Loss with multiple positive/negative visual examples, rather than relying solely on…

A vulnerability exists in the tool selection mechanism of Large Language Model (LLM) agents that utilize a retrieval-then-selection pipeline (RAG) for identifying executable tools. The vulnerability, known as "ToolHijacker," allows a remote attacker to manipulate the agent's decision-making process by injecting a malicious tool document into the accessible tool library (e.g., via third-party tool hubs or plugins). The attack employs a two-phase optimization strategy to craft a malicious tool…

Retrieval-Augmented Generation (RAG) systems are vulnerable to a targeted corpus poisoning attack known as "CorruptRAG". This vulnerability allows an attacker to manipulate the response of an LLM to a specific target query by injecting a single malicious document into the RAG knowledge database. Unlike traditional poisoning attacks that require flooding the retrieval results (top-N) with malicious content to outnumber correct information, CorruptRAG succeeds with a single retrieved document.

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

Multimodal Large Language Models (MLLMs) are vulnerable to a universal adversarial attack where a single, optimized image can bypass safety alignment mechanisms across diverse textual queries. By employing gradient-based optimization on the input image pixels while propagating gradients through the vision encoder and language model, an attacker can craft a visual perturbation that coerces the model into a compliant state. When this adversarial image is present in the context, the model’s…

Retrieval-Augmented Generation (RAG) systems utilizing dense retrieval mechanisms are vulnerable to topic-oriented adversarial corpus poisoning, specifically via the "Topic-FlipRAG" attack method. This vulnerability allows an attacker to manipulate the opinion or stance of the LLM's output across a broad cluster of related queries, rather than a single specific prompt. The attack leverages a two-stage pipeline: (1) Knowledge-Guided Attack, where an LLM is used to edit a target document to…

A vulnerability exists in Large Language Model (LLM) routing systems (control planes) that allows for the manipulation of inference flow via adversarial input sequences. LLM routers, which dynamically direct user queries to either "weak" (cheaper) or "strong" (expensive) models based on predicted query complexity, can be bypassed by appending specific, pre-optimized token sequences known as "confounder gadgets." These gadgets artificially inflate the router's complexity score for an input…

A vulnerability in LLM-based agents, dubbed AI Agent Injection (AI²), allows attackers to hijack the agent's actions by manipulating the agent's memory retrieval mechanism. The attack involves two main steps: (1) Stealing action-aware knowledge from the agent's memory using crafted adversarial queries targeting the retriever module and (2) Generating Trojan prompts consisting of a Trojan string and hijacking instructions. The Trojan string is designed to manipulate the retriever into…