Prompt layer Vulnerabilities

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

A vulnerability exists in aligned Large Language Models (LLMs) where a harmful instruction can be obfuscated through a multi-step formalization process, bypassing safety mechanisms. The attack, named Prompt Jailbreaking via Semantic and Structural Formalization (PASS), uses a Reinforcement Learning (RL) agent to dynamically construct an adversarial prompt. The agent learns to apply a sequence of actions—such as symbolic abstraction, logical encoding, mathematical representation, metaphorical…

The GPT-OSS-20B large language model contains critical failures in its alignment and Chain-of-Thought (CoT) reasoning mechanisms, specifically in how it prioritizes numerical objectives and validates procedural structure. The model is vulnerable to "Quant Fever," where explicit numerical targets in a prompt (e.g., "delete 90% of files") override contextual safety constraints (e.g., "do not delete important files"). Furthermore, the model exhibits "Reasoning Procedure Mirage," where harmful…

A vulnerability exists in Large Language Models (LLMs) and multi-label text classification systems that allows for Textual Dynamic Outputs Attacks (TDOA). This technique enables hard-label black-box attacks against systems with variable or generative output spaces (where the number of labels or specific label tokens are not fixed). The attack functions by training a surrogate model on clustered coarse-grained labels derived from the victim model's fine-grained dynamic outputs. It subsequently…

A vulnerability exists in multiple Large Language Models (LLMs) that allows for safety alignment bypass through a technique named Activation-Guided Local Editing (AGILE). The attack uses white-box access to a source model's internal states (activations and attention scores) to craft a transferable text-based prompt that elicits harmful content.

Large Language Models (LLMs) and Vision-Language Models (VLMs) are vulnerable to an automated, adaptive role-play jailbreak attack known as GUARD (Guideline Upholding Test through Adaptive Role-play and Jailbreak Diagnostics). The vulnerability exists because the models fail to recognize malicious intent when harmful queries are embedded within complex, iteratively optimized "playing scenarios."

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Large Language Models (LLMs) exposed via public APIs are vulnerable to model fingerprinting attacks where an attacker can identify the exact backend model family and version (e.g., distinguishing Mistral-7B-v0.1 from v0.3) by analyzing response patterns. While traditional fingerprinting relies on manual query curation, this vulnerability is exacerbated by Reinforcement Learning (RL) based query optimization. An attacker can train an RL agent (specifically using Proximal Policy Optimization) to…

Large Language Models (LLMs) are vulnerable to automated adversarial attacks that systematically combine multiple jailbreaking "primitives" into complex prompt chains. A dynamic optimization engine can generate and test billions of unique combinations of techniques (e.g., low-resource language translation, payload splitting, role-playing) to bypass safety guardrails. This combinatorial approach differs from manual red-teaming by systematically exploring the attack surface, achieving…

Large Reasoning Models (LRMs) can be instructed via a single system prompt to act as autonomous adversarial agents. These agents engage in multi-turn persuasive dialogues to systematically bypass the safety mechanisms of target language models. The LRM autonomously plans and executes the attack by initiating a benign conversation and gradually escalating the harmfulness of its requests, thereby circumventing defenses that are not robust to sustained, context-aware persuasive attacks. This…