Attack Surface

Prompt layer Research

Security research involving prompt handling and processing

610 matching entries

Prompt layer research results

610 entries

Meta-Optimized LLM Judge Jailbreak

Published Nov 3, 2025Analyzed Nov 20, 2025Source: arXiv

A vulnerability in Large Language Models (LLMs) allows for systematic jailbreaking through a meta-optimization framework called AMIS (Align to MISalign). The attack uses a bi-level optimization process to co-evolve both the jailbreak prompts and the scoring templates used to evaluate them.

Align to Misalign: Automatic LLM Jailbreak with Meta-Optimized LLM Judges

Evaluated models: Claude 3.5 Haiku, Claude 3.5 Sonnet, Claude Sonnet 4+3 more

Claude 3.5 Haiku
Claude 3.5 Sonnet
Claude Sonnet 4
GPT-4o
GPT-4o Mini
Llama 3.1 8B Instruct

Adversarial Self-Deception

Published Oct 31, 2025Analyzed Dec 30, 2025Source: arXiv

Large Language Models (LLMs) exhibit a vulnerability to "adversarial conversational nudges," where the model abandons its internal factual knowledge to align with user-provided misinformation in closed domains (e.g., movies, books). Unlike standard hallucinations where a model lacks knowledge, this vulnerability occurs even when the model demonstrates—via separate self-consistency checks—that it correctly identifies the information as false. When a user creates a multi-turn context asserting…

What About the Scene With the Hitler Reference? HAUNT: A Framework to Probe LLMs' Self-consistency in Closed Domains Via Adversarial Nudge

Evaluated models: GPT-4o, GPT-5, Claude Opus 4+4 more

GPT-4o
GPT-5
Claude Opus 4
Gemini 1.5 Flash
Gemini 2.5 Flash
DeepSeek Reasoner
Grok 4

LLM Self-Harm Loop

Published Oct 31, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs), specifically GPT-3.5-turbo, LLaMA3-8B-instruct, and DeepSeek-R1-Distill-Qwen-7B, are vulnerable to a "Self-Harm" jailbreak attack (Self-HarmLLM). This vulnerability exploits the model's ability to understand its own safety boundaries to generate adversarial inputs against itself. An attacker utilizes a two-session approach: in the first session (Mitigation Session), the attacker instructs the model to rewrite a harmful query into a "Mitigated Harmful Query"…

Self-HarmLLM: Can Large Language Model Harm Itself?

Evaluated models: GPT-3.5 Turbo, Llama 3 8B Instruct, DeepSeek R1 Distill Qwen 7B

Mobile Agent Channel Subversion

Published Oct 31, 2025Analyzed Dec 30, 2025Source: arXiv

Mobile LLM-based agents (including Mobile-Agent-E, AppAgent, AutoDroid, and others) are vulnerable to indirect prompt injection attacks delivered via untrusted third-party mobile channels, such as in-app advertisements, system notifications, and embedded webviews. These agents utilize Multimodal Large Language Models (MLLMs) to perceive the device state via screenshots or accessibility trees. The vulnerability exists because the agents concatenate the user's prompt ($p$) with the environmental…

Measuring the Security of Mobile LLM Agents under Adversarial Prompts from Untrusted Third-Party Channels

Evaluated models: GPT-3.5 Turbo, GPT-4 Turbo, GPT-4o+1 more

GPT-3.5 Turbo
GPT-4 Turbo
GPT-4o
GPT-4.1 Mini

Latent Paraphrase Segmentation Attack

Published Oct 28, 2025Analyzed Dec 30, 2025Source: arXiv

Reasoning segmentation models, which generate binary segmentation masks based on implicit text queries, are vulnerable to adversarial paraphrasing. This vulnerability allows an attacker to craft semantically equivalent and grammatically correct text prompts that significantly degrade the model's segmentation performance (measured by Intersection-over-Union, or IoU). The exploit utilizes a black-box, sentence-level optimization method (SPARTA) that operates within the continuous semantic latent…

SPARTA: Evaluating Reasoning Segmentation Robustness through Black-Box Adversarial Paraphrasing in Text Autoencoder Latent Space

Evaluated models: LISA 7B, LISA Explanatory 7B, LISA 13B+3 more

LISA 7B
LISA Explanatory 7B
LISA 13B
LISA Explanatory 13B
LISA++ 7B
GSVA 13B

Internal Prompt Leak Injection

Published Oct 27, 2025Analyzed Dec 30, 2025Source: arXiv

LLM-based coding agents integrated into IDEs (e.g., VS Code Copilot, Cursor, Windsurf) are vulnerable to a query-agnostic Indirect Prompt Injection (IPI) attack termed "QueryIPI." This vulnerability allows an attacker to achieve Remote Code Execution (RCE) on the developer's machine by injecting a malicious tool definition (e.g., via the Model Context Protocol) into the agent's context.

QueryIPI: Query-agnostic Indirect Prompt Injection on Coding Agents

Evaluated models: Claude Sonnet 4

Concurrent Task Jailbreak

Published Oct 24, 2025Analyzed Nov 20, 2025Source: arXiv

A jailbreak vulnerability, known as Task Concurrency, exists in multiple Large Language Models (LLMs). The vulnerability arises when two distinct tasks, one harmful and one benign, are interleaved at the word level within a single prompt. The structure of the malicious prompt alternates words from each task, often using separators like {} to encapsulate words from the second task. This "concurrent" instruction format obfuscates the harmful intent from the model's safety guardrails, causing the…

Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency

Evaluated models: DeepSeek V3, Gemini 2.5 Flash, GPT-4.1+7 more

DeepSeek V3
Gemini 2.5 Flash
GPT-4.1
GPT-4o
GPT-4o Mini
Llama 2 13B
Llama 2 7B
Llama 3 8B
Mistral 7B
Vicuna 13B

Persuasive Jailbreak Fingerprint

Published Oct 24, 2025Analyzed Nov 1, 2025Source: arXiv

Large Language Models (LLMs) are vulnerable to jailbreak attacks that use persuasive techniques grounded in social psychology to bypass safety alignments. Malicious instructions can be reframed using one of Cialdini's seven principles of persuasion (Authority, Reciprocity, Commitment, Social Proof, Liking, Scarcity, and Unity). These rephrased prompts, which remain human-readable and can be generated automatically, manipulate the LLM into complying with harmful requests it would otherwise…

Uncovering the Persuasive Fingerprint of LLMs in Jailbreaking Attacks

Evaluated models: DeepSeek R1, GPT-2, Phi-4+1 more

DeepSeek R1
GPT-2
Phi-4
WizardLM Uncensored

LLM Data Instruction Override

Published Oct 22, 2025Analyzed Dec 30, 2025Source: arXiv

Large Language Model (LLM) integrated agents and applications are vulnerable to Prompt Injection attacks where untrusted data (e.g., retrieved documents, tool outputs, website content) overrides system instructions. Because LLMs typically process instructions and data within a single context window without strict separation, an attacker can embed imperative commands within the data channel. This vulnerability extends beyond simple overriding instructions; it includes sophisticated techniques…

Defending against prompt injection with datafilter

Evaluated models: GPT-4o, Llama 3.1 8B Instruct

Adaptive Traversal Jailbreak

Published Oct 21, 2025Analyzed Dec 8, 2025Source: arXiv

Large Language Models (LLMs) including GPT-4o, LLaMA-3, and Mistral-7B are vulnerable to an adaptive multi-turn jailbreak attack known as HarmNet. This vulnerability exploits the model's inability to detect malicious intent when it is distributed across a hierarchical semantic network (ThoughtNet) rather than a single prompt. The attack methodology involves three phases: (1) constructing a semantic network of candidate topics and contextual sentences using embedding similarity to obscure the…

A Framework for Adaptive Multi-Turn Jailbreak Attacks on Large Language Models

Evaluated models: GPT-3.5 Turbo, GPT-4o, Claude 3.5 Sonnet+3 more

GPT-3.5 Turbo
GPT-4o
Claude 3.5 Sonnet
Llama 3 8B
Mistral 7B
Gemma 2 9B

Related research categories