Attack Surface

Application layer Research

Security research involving application interfaces and API implementations

16 matching entries out of 273 in this category

Application layer research results

16 entries

Long-Horizon Agent Attacks

Published Feb 18, 2026Analyzed Apr 10, 2026Source: arXiv

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks

Evaluated models: GPT-4o, GPT-5.1, Gemini 3 Flash+1 more

GPT-4o
GPT-5.1
Gemini 3 Flash
Claude Sonnet 4.5

Autonomous Agent Prompt Reveal

Published Jan 29, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs

Evaluated models: o1, Llama 3.1 70B Hanami X1, Phi-4+38 more

o1
Llama 3.1 70B Hanami X1
Phi-4
Aion 1.0
Sonar Pro
Command A
Llama 4 Maverick
Llama 3.1 Nemotron Ultra 253B v1
Qwen 3 235B-A22B
Mercury
Hunyuan A13B Instruct
UI-TARS 1.5 7B
GPT-oss 120B
Jamba Mini 1.7
Hermes 4 70B
Step 3
LongCat Flash Chat
Tongyi DeepResearch 30B-A3B
Cydonia 24B v4.1
ERNIE 4.5 21B-A3B Thinking
Granite 4.0 H Micro
LFM2 8B-A1B
Nova Premier v1
Kimi K2 Thinking
KAT Coder Pro
Cogito v2.1 671B
Gemini 3 Pro Preview
Grok 4.1 Fast
Claude Opus 4.5
TNG R1T Chimera
Intellect 3
DeepSeek V3.2 Speciale
Trinity Mini
Mistral Large 2512
DeepSeek V3.1 Nex N1
MiMo V2 Flash
GLM-4.7
MiniMax M2.1
Seed 1.6
Molmo 2 8B
GPT-5.2 Codex

Dual Stego MLLM Jailbreak

Published Dec 23, 2025Analyzed Dec 30, 2025Source: arXiv

Commercial Multimodal Large Language Model (MLLM) integrated systems are vulnerable to a "Dual Steganography" jailbreak paradigm (referred to as Odysseus). The vulnerability arises from the reliance of safety filters on the assumption that malicious content must be explicitly visible in the input or output modalities (text or image). Attackers can bypass these filters by encoding malicious queries into binary matrices and embedding them into benign-looking images using steganographic encoders…

Odysseus: Jailbreaking Commercial Multimodal LLM-integrated Systems via Dual Steganography

Evaluated models: GPT-4o, Gemini 2.0 Pro, Gemini 2.0 Flash+1 more

GPT-4o
Gemini 2.0 Pro
Gemini 2.0 Flash
Grok 3

Special Token Jailbreak

Published Oct 11, 2025Analyzed Oct 31, 2025Source: arXiv

Large Language Models (LLMs) that use special tokens to define conversational structure (e.g., via chat templates) are vulnerable to a jailbreak attack named MetaBreak. An attacker can inject these special tokens, or regular tokens with high semantic similarity in the embedding space, into a user prompt. This manipulation allows the attacker to bypass the model's internal safety alignment and external content moderation systems. The attack leverages four primitives: 1. Response Injection…

MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation

Evaluated models: Claude Opus 4, Gemma 2 27B IT, GPT-4.1+9 more

Claude Opus 4
Gemma 2 27B IT
GPT-4.1
Llama 3.1 405B
Llama 3.1 8B
Llama 3.3 70B Instruct
Llama Guard
Llama Guard 3 8B
Phi-4
Prompt Guard
Qwen 2.5 72B Instruct
ShieldGemma 2 27B

Typos Undermine Watermarks

Published Sep 11, 2025Analyzed Feb 22, 2026Source: arXiv

Large Language Model (LLM) inference-time watermarking schemes are vulnerable to evasion via character-level perturbations that disrupt the model's tokenizer. Unlike token-level attacks (e.g., synonym replacement), character-level edits—such as homoglyph substitutions, zero-width character insertions, and typos—force the tokenizer to segment a single semantic unit into multiple sub-word tokens. This fragmentation alters the context window used by the watermarking hashing function (e.g., the…

Character-Level Perturbations Disrupt LLM Watermarks

Evaluated models: Llama 3 8B

Multimodal Prompt Injection

Published Sep 7, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Evaluated models: GPT-3.5, GPT-4o, Llama 3 8B+1 more

GPT-3.5
GPT-4o
Llama 3 8B
Mistral Large 24B

MDH: Hybrid Jailbreak Detection Strategy

Published Aug 14, 2025Analyzed Aug 31, 2025Source: arXiv

Large language models that support a developer role in their API are vulnerable to a jailbreaking attack that leverages malicious developer messages. An attacker can craft a developer message that overrides the model's safety alignment by setting a permissive persona, providing explicit instructions to bypass refusals, and using few-shot examples of harmful query-response pairs. This technique, named D-Attack, is effective on its own. A more advanced variant, DH-CoT, enhances the attack by…

Jailbreaking Commercial Black-Box LLMs with Explicitly Harmful Prompts

Evaluated models: GPT-3.5 Turbo, GPT-4o, GPT-4.1+13 more

GPT-3.5 Turbo
GPT-4o
GPT-4.1
o1-mini
o1
o3-mini
o3
o4-mini
Gemini 2.5 Pro
Gemini 2.0 Flash Thinking
Claude 3.5 Sonnet
Claude 3.7 Sonnet
Claude Sonnet 4
DeepSeek V3
DeepSeek R1 0528
DeepSeek R1

Agent Tool Metadata Lure

Published Aug 4, 2025Analyzed Feb 22, 2026Source: arXiv

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools

Evaluated models: GPT-4o Mini, Llama 3.3 70B Instruct, Qwen 2.5 32B Instruct+2 more

GPT-4o Mini
Llama 3.3 70B Instruct
Qwen 2.5 32B Instruct
Gemma 3 27B IT
Qwen 3 32B

LLM Interpreter Resource Exhaustion

Published Jul 25, 2025Analyzed Aug 16, 2025Source: arXiv

Large Language Models (LLMs) equipped with native code interpreters are vulnerable to Denial of Service (DoS) via resource exhaustion. An attacker can craft a single prompt that causes the interpreter to execute code that depletes CPU, memory, or disk resources. The vulnerability is particularly pronounced when a resource-intensive task is framed within a plausibly benign or socially-engineered context ("indirect prompts"), which significantly lowers the model's likelihood of refusal compared…

Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security

Evaluated models: Gemini 2.0 Flash, Gemini 2.5 Flash, Gemini 2.5 Pro+5 more

Gemini 2.0 Flash
Gemini 2.5 Flash
Gemini 2.5 Pro
GPT-4.1
GPT-4.1 Mini
GPT-4.1 Nano
o3 Pro
o4-mini

Schema-Guided LLM Jailbreak

Published Mar 31, 2025Analyzed Apr 12, 2025Source: arXiv

Large Language Models (LLMs) with structured output APIs (e.g., using JSON Schema) are vulnerable to Constrained Decoding Attacks (CDAs). CDAs exploit the control plane of the LLM's decoding process by embedding malicious intent within the schema-level grammar rules, bypassing safety mechanisms that primarily focus on input prompts. The attack manipulates the allowed output space, forcing the LLM to generate harmful content despite a benign input prompt. One instance of a CDA is the Chain Enum…

Output Constraints as Attack Surface: Exploiting Structured Generation to Bypass LLM Safety Mechanisms

Evaluated models: Gemini 2.0 Flash, Gemma 2 9B, GPT-4o+5 more

Gemini 2.0 Flash
Gemma 2 9B
GPT-4o
GPT-4o Mini
Llama 3.1 8B
Mistral Nemo
Phi 3.5 MoE
Qwen 2.5 32B

Related research categories