Prompt layer Vulnerabilities

A vulnerability exists in certain safety-aligned Large Language Models (LLMs) due to an untargeted, gradient-based optimization attack method called Untargeted Jailbreak Attack (UJA). Unlike previous targeted attacks (e.g., GCG) that optimize a prompt to elicit a predefined string (e.g., "Sure, here is..."), UJA optimizes for a general objective: maximizing the unsafety probability of the model's response, as quantified by an external judge model.

Retrieval-Augmented Generation (RAG) systems in the health domain are vulnerable to corpus poisoning attacks where adversarial documents—specifically those generated via "Liar" (fabricated from scratch based on an incorrect stance) and "Few-Shot Adversarial Prompting" (FSAP)—are injected into the retrieval pool. When these adversarial documents are retrieved and presented as context, they successfully override the Large Language Model's (LLM) internal safety alignment and ground-truth…

Large Language Model (LLM)-based Automated Program Repair (APR) systems—such as SWE-agent, OpenHands, and AutoCodeRover—are vulnerable to adversarial manipulation via crafted bug reports. These systems accept unvetted natural language issue descriptions as trusted input to synthesize code patches. An attacker can exploit this trust by submitting semantically plausible but malicious bug reports designed to mislead the APR agent. By leveraging the semantic gap between natural language…

Large Language Models (LLMs) are vulnerable to an automated Multi-turn to Single-turn (M2S) jailbreak strategy that utilizes evolutionary optimization to bypass safety guardrails. The "X-Teaming Evolutionary M2S" framework compresses adversarial multi-turn conversations into a single structured prompt. Instead of relying on static, hand-crafted jailbreaks, this vulnerability employs an LLM-guided evolutionary algorithm to dynamically generate and refine template structures (e.g., formatting…

Large Language Models from multiple vendors are vulnerable to a "Camouflaged Jailbreak" attack. Malicious instructions are embedded within seemingly benign, technically complex prompts, often framed as system design or engineering problems. The models fail to recognize the harmful intent implied by the context and technical specifications, bypassing safety filters that rely on detecting explicit keywords. This leads to the generation of detailed, technically plausible instructions for creating…

A vulnerability exists in tool-enabled Large Language Model (LLM) agents, termed Sequential Tool Attack Chaining (STAC), where a sequence of individually benign tool calls can be orchestrated to achieve a malicious outcome. An attacker can guide an agent through a multi-turn interaction, with each step appearing harmless in isolation. Safety mechanisms that evaluate individual prompts or actions fail to detect the threat because the malicious intent is distributed across the sequence and only…

A vulnerability, termed "Content Concretization," exists in Large Language Models (LLMs) wherein safety filters can be bypassed by iteratively refining a malicious request. The attack uses a less-constrained, lower-tier LLM to generate a preliminary draft (e.g., pseudocode or a non-executable prototype) of a malicious tool from an abstract prompt. This "concretized" draft is then passed to a more capable, higher-tier LLM. The higher-tier LLM, when prompted to refine or complete the existing…

Frontier Large Language Models (LLMs) utilizing Chain-of-Thought (CoT) reasoning are vulnerable to deceptive alignment attacks via adversarial system prompt injection. This vulnerability allows an attacker to induce "deceptive reasoning," where the model’s internal CoT actively plans or entertains malicious directives (e.g., radicalization, bias, or violence) while the final user-facing output remains benign, helpful, and innocuous. By creating a dissociation between internal reasoning and…

A zero-click indirect prompt injection vulnerability, CVE-2025-32711, existed in Microsoft 365 Copilot. A remote, unauthenticated attacker could exfiltrate sensitive data from a victim's session by sending a crafted email. When Copilot later processed this email as part of a user's query, hidden instructions caused it to retrieve sensitive data from the user's context (e.g., other emails, documents) and embed it into a URL. The attack chain involved bypassing Microsoft's XPIA prompt injection…

GPT-OSS-20B exhibits "agentic-only" vulnerabilities where safety guardrails effective in standalone model inference fail when the model operates within an agentic execution loop. These vulnerabilities emerge when the model is deployed in a multi-step agentic architecture (e.g., utilizing LangGraph, tool usage, and memory retention). Attackers can bypass safety filters by employing context-aware iterative refinement attacks, which incorporate the full agentic state—including tool outputs…