Language Model Security Database

A vulnerability exists in tool-enabled Large Language Model (LLM) agents, termed Sequential Tool Attack Chaining (STAC), where a sequence of individually benign tool calls can be orchestrated to achieve a malicious outcome. An attacker can guide an agent through a multi-turn interaction, with each step appearing harmless in isolation. Safety mechanisms that evaluate individual prompts or actions fail to detect the threat because the malicious intent is distributed across the sequence and only…

Frontier Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) reasoning are vulnerable to deceptive alignment attacks via adversarial system prompt injection. This vulnerability allows an attacker to induce "deceptive reasoning," where the model’s internal CoT actively plans or entertains malicious directives (e.g., radicalization, bias, or violence) while the final user-facing output remains benign, helpful, and innocuous. By creating a dissociation between internal reasoning and…

A vulnerability exists in multiple Large Language Models (LLMs) where an attacker can bypass safety alignments by exploiting the model's ethical reasoning capabilities. The attack, named TRIAL (Trolley-problem Reasoning for Interactive Attack Logic), frames a harmful request within a multi-turn ethical dilemma modeled on the trolley problem. The harmful action is presented as the "lesser of two evils" necessary to prevent a catastrophic outcome, compelling the model to engage in utilitarian…

A vulnerability exists in multiple Large Language Models (LLMs) where safety alignment mechanisms can be bypassed by reframing harmful instructions as "learning-style" or academic questions. This technique, named Hiding Intention by Learning from LLMs (HILL), transforms direct, harmful requests into exploratory questions using simple hypotheticality indicators (e.g., "for academic curiosity", "in the movie") and detail-oriented inquiries (e.g., "provide a step-by-step breakdown"). The attack…

Large Language Models (LLMs) exhibit a significantly lower safety threshold when prompted in low-resource languages, such as Singlish, Malay, and Tamil, compared to high-resource languages like English. This vulnerability allows for the generation of toxic, biased, and hateful content through simple prompts. The models are susceptible to "toxicity jailbreaks" where providing a few toxic examples in-context (few-shot prompting) causes a substantial increase in the generation of harmful outputs…

The evaluated MetaGPT multi-agent systems are vulnerable to "Web Fraud Attacks" due to insufficient semantic and structural validation of Uniform Resource Locators (URLs) by agentic models. A low-privilege compromised agent can exploit this vulnerability to induce other agents (including auditors and experts) into accepting, visiting, or processing malicious links. The vulnerability leverages the LLM's inability to distinguish between benign and malicious link structures when obfuscation…

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Large Language Models (LLMs) employed as automated assistants or autonomous agents in academic peer review systems are vulnerable to indirect prompt injection via maliciously crafted PDF submissions. Attackers can embed adversarial instructions within the manuscript that are invisible to human reviewers (using techniques such as white-on-white text or manipulating TrueType font character mapping tables) but are parsed and executed by the LLM.

AdvEDM reveals a vulnerability in Vision-Language Model (VLM) based Embodied Decision-Making (EDM) systems, such as those used in autonomous driving and robotic manipulation. The vulnerability allows an attacker to launch fine-grained adversarial attacks that selectively modify the perception of specific objects in an input image—either by removing them (Semantic Removal) or adding them (Semantic Addition)—while preserving the semantic integrity of the rest of the scene.

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…