Language Model Security Database

A vulnerability exists in Large Language Model (LLM) agents that allows attackers to manipulate the agent's reasoning process through the insertion of strategically placed adversarial strings. This allows attackers to induce the agent to perform unintended malicious actions or invoke specific malicious tools, even when the initial prompt or instruction is benign. The attack exploits the agent's reliance on chain-of-thought reasoning and dynamically optimizes the adversarial string to maximize…

FC-Attack leverages automatically generated flowcharts containing step-by-step descriptions derived or rephrased from harmful queries, combined with a benign textual prompt, to jailbreak Large Vision-Language Models (LVLMs). The vulnerability lies in the model's susceptibility to visual prompts containing harmful information within the flowcharts, thus bypassing safety alignment mechanisms.

LLM agents utilizing external tools are vulnerable to indirect prompt injection (IPI) attacks. Attackers can embed malicious instructions into the external data accessed by the agent, manipulating its behavior even when defenses against direct prompt injection are in place. Adaptive attacks, which modify the injected payload based on the specific defense mechanism, consistently bypass existing defenses with a success rate exceeding 50%.

Commercial LLM-powered agents utilizing autonomous web access, memory modules, and retrieval-augmented generation (RAG) are vulnerable to indirect prompt injection and environmental manipulation. Attackers can embed malicious instructions into external data sources trusted by the agent (such as Reddit posts, public databases, or ArXiv papers). When the agent autonomously retrieves and processes this content during task execution, it executes the embedded malicious commands. This vulnerability…

The Virus attack method enables attackers to bypass guardrail moderation on fine-tuning data, leading to a significant degradation of safety alignment in large language models (LLMs). This is achieved through a dual-objective data optimization strategy that crafts harmful data undetectable by the guardrail while maximizing their effectiveness in compromising the victim model's safety.

Vision Language Models (VLMs) integrated into autonomous driving (AD) systems are vulnerable to a black-box adversarial attack method termed Cascading Adversarial Disruption (CAD). The vulnerability stems from the model's susceptibility to optimized visual perturbations that disrupt the decision-making reasoning chain (perception, prediction, and planning). Attackers can generate adversarial images or physical patches by aligning visual noise with deceptive textual semantics in the model's…

Vision Language Models (VLMs) are vulnerable to visual prompt injection attacks via text-to-image obfuscation. While these models often possess safety guardrails for standard text-based inputs, they fail to apply equivalent safety alignment to textual instructions embedded visually within an image. An attacker can overlay malicious instructions (e.g., requests for illegal acts, hate speech) onto an image file and submit it to the model. The model’s Optical Character Recognition (OCR) or visual…

Large Language Model (LLM) tool-calling systems are vulnerable to adversarial tool injection attacks. Attackers can inject malicious tools ("Manipulator Tools") into the tool platform, manipulating the LLM's tool selection and execution process. This allows for privacy theft (extracting user queries), denial-of-service (DoS) attacks against legitimate tools, and unscheduled tool-calling (forcing the use of attacker-specified tools regardless of relevance). The attack exploits vulnerabilities…

Large Language Model (LLM) agents are vulnerable to obfuscated adversarial prompts that exploit tool misuse. These prompts, crafted through prompt optimization techniques, force the agent to execute tools (e.g., URL fetching, markdown rendering) in a way that leaks sensitive user data (e.g., PII) without the user's knowledge. The prompts are designed to be visually indistinguishable from benign prompts.

Jailbreaking vulnerabilities in Large Language Models (LLMs) used in Retrieval-Augmented Generation (RAG) systems allow escalation of attacks from entity extraction to full document extraction and enable the propagation of self-replicating malicious prompts ("worms") within interconnected RAG applications. Exploitation leverages prompt injection to force the LLM to return retrieved documents or execute malicious actions specified within the prompt.