Attack Surface

Prompt layer Research

Security research involving prompt handling and processing

610 matching entries

Prompt layer research results

610 entries

Emergent Character Misalignment

Published Jan 30, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs), specifically Llama-3.1-8B-Instruct and Qwen2.5-14B-Instruct, are vulnerable to emergent misalignment caused by "character-conditioned" fine-tuning. This vulnerability arises when models are fine-tuned on small datasets (e.g., 500 examples) that exhibit consistent behavioral dispositions (e.g., "Evil," "Sycophantic," or "Hallucinatory") rather than just incorrect facts. This process creates a latent control variable—defined as "character"—that governs model…

Character as a Latent Variable in Large Language Models: A Mechanistic Account of Emergent Misalignment and Conditional Safety Failures

Evaluated models: GPT-5, Llama 3.1 8B, Qwen 2.5 14B

Payment Protocol Whisper Attack

Published Jan 30, 2026Analyzed Feb 21, 2026Source: arXiv

The Google Agent Payments Protocol (AP2), specifically within the reference implementation built using the Google Agent Development Kit (ADK) and Gemini models, contains vulnerabilities allowing for both indirect and direct prompt injection. The architecture fails to sufficiently isolate the Large Language Model (LLM) context from untrusted external data sources and user inputs.

Whispers of Wealth: Red-Teaming Google's Agent Payments Protocol via Prompt Injection

Evaluated models: Gemini 2.5 Flash

Semantics-Preserving Detector Evasion

Published Jan 30, 2026Analyzed Feb 21, 2026Source: arXiv

LLM-based vulnerability detection systems (used in static application security testing and code review pipelines) are susceptible to semantics-preserving adversarial evasion attacks. Attackers can bypass detection mechanisms by injecting gradient-optimized "universal adversarial strings" into specific code regions—defined as "carriers"—that do not alter the program's compilation or execution logic. These carriers include non-executable regions (code comments, inactive preprocessor directives)…

Syntax- and Compilation-Preserving Evasion of LLM Vulnerability Detectors

Evaluated models: Qwen 2.5 Coder 14B, Qwen 2.5 Coder 32B, Llama 3.1 8B+4 more

Qwen 2.5 Coder 14B
Qwen 2.5 Coder 32B
Llama 3.1 8B
CodeAstra
StarCoder2 15B
GPT-4o
GPT-5 Mini

Autonomous Agent Prompt Reveal

Published Jan 29, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in Large Language Model (LLM) deployments and multi-agent systems where an autonomous attacker agent can systematically extract hidden system prompts through self-evolving interaction strategies. The vulnerability leverages a "JustAsk" framework which utilizes Upper Confidence Bound (UCB) exploration to dynamically select and refine attack vectors from a hierarchical taxonomy of 14 atomic skills (e.g., structural formatting, authority appeals) and 14 multi-turn…

Just Ask: Curious Code Agents Reveal System Prompts in Frontier LLMs

Evaluated models: o1, Llama 3.1 70B Hanami X1, Phi-4+38 more

o1
Llama 3.1 70B Hanami X1
Phi-4
Aion 1.0
Sonar Pro
Command A
Llama 4 Maverick
Llama 3.1 Nemotron Ultra 253B v1
Qwen 3 235B-A22B
Mercury
Hunyuan A13B Instruct
UI-TARS 1.5 7B
GPT-oss 120B
Jamba Mini 1.7
Hermes 4 70B
Step 3
LongCat Flash Chat
Tongyi DeepResearch 30B-A3B
Cydonia 24B v4.1
ERNIE 4.5 21B-A3B Thinking
Granite 4.0 H Micro
LFM2 8B-A1B
Nova Premier v1
Kimi K2 Thinking
KAT Coder Pro
Cogito v2.1 671B
Gemini 3 Pro Preview
Grok 4.1 Fast
Claude Opus 4.5
TNG R1T Chimera
Intellect 3
DeepSeek V3.2 Speciale
Trinity Mini
Mistral Large 2512
DeepSeek V3.1 Nex N1
MiMo V2 Flash
GLM-4.7
MiniMax M2.1
Seed 1.6
Molmo 2 8B
GPT-5.2 Codex

LLM Grading Compliance Paradox

Published Jan 29, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) employed as automated code evaluators ("Universal Graders") are vulnerable to Semantic-Instruction Decoupling, a form of adversarial prompt injection that exploits the "Syntax-Semantics Gap." Attackers can embed adversarial directives into syntactically inert regions of the Abstract Syntax Tree (AST)—specifically comments, docstrings, variable names, and whitespace. While these regions are discarded by compilers (trivia nodes) or treated as arbitrary symbols…

The Compliance Paradox: Semantic-Instruction Decoupling in Automated Academic Code Evaluation

Evaluated models: GPT-5, Llama 3.1 8B, DeepSeek V3

LLM Router Rerouting

Published Jan 29, 2026Analyzed Mar 8, 2026Source: arXiv

LLM routing systems are vulnerable to adversarial rerouting attacks where malicious triggers prepended to user queries manipulate the router's model-selection mechanism. Because LLM routers function as classifiers evaluating query complexity to balance computational cost and response quality, an attacker can craft adversarial prefixes that distort the query's latent semantic representation. This exploits the router's decision boundaries, forcing the system to misclassify the input and redirect…

RerouteGuard: Understanding and Mitigating Adversarial Risks for LLM Routing

Evaluated models: GPT-4, GPT-4o, GPT-5+2 more

GPT-4
GPT-4o
GPT-5
Llama 3 8B
Mixtral 8x7B

Pathological Reasoning DoS

Published Jan 29, 2026Analyzed Mar 9, 2026Source: arXiv

A vulnerability in Large Reasoning Models (LRMs) allows attackers to perform Prompt-Induced Inference-Time Denial-of-Service (PI-DoS) attacks by submitting short, semantically coherent adversarial prompts. These prompts, which often take the form of complex logic puzzles with nested dependencies or contradictory constraints, exploit the adaptive computation mechanism of LRMs to force the model into pathologically long, nearly non-terminating intermediate reasoning traces (e.g., generating…

ReasoningBomb: A Stealthy Denial-of-Service Attack by Inducing Pathologically Long Reasoning in Large Reasoning Models

Evaluated models: DeepSeek V3, Kimi K2 Instruct, DeepSeek R1+8 more

DeepSeek V3
Kimi K2 Instruct
DeepSeek R1
DeepSeek R1 Distill Qwen 32B
MiniMax M2
Nemotron 3 Nano 30B-A3B
Qwen 3 30B-A3B Thinking
Qwen 3 32B
Claude Sonnet 4.5
GPT-5
Gemini 3 Pro Preview

Persona Adherence Jailbreak

Published Jan 29, 2026Analyzed Mar 8, 2026Source: arXiv

Large Language Models (LLMs) deployed as Role-Playing Agents (RPAs) are vulnerable to Persona-Targeted Jailbreaks. When an LLM is prompted to adopt a persona—particularly characters with negative, risky, or villainous traits—the model's optimization for role fidelity frequently overrides its foundational safety alignment. Attackers can exploit this vulnerability by synthesizing queries that leverage the specific narrative background, ideology, or psychological vulnerabilities of the assigned…

Stay in Character, Stay Safe: Dual-Cycle Adversarial Self-Evolution for Safety Role-Playing Agents

Evaluated models: GPT-4o, Llama 3 8B, Qwen 2.5 7B+1 more

GPT-4o
Llama 3 8B
Qwen 2.5 7B
Gemma 2 9B

Intent-Context Coupling Jailbreak

Published Jan 28, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs), including GPT-4o, Claude 3.5 Sonnet, and Llama 3, are vulnerable to an "Intent-Context Coupling" multi-turn jailbreak attack (automated by the ICON framework). The vulnerability arises from an alignment failure where safety constraints are relaxed when a malicious intent is paired with a semantically congruent "authoritative-style" context pattern. By routing specific prohibited intents (e.g., Hacking) to pre-optimized context patterns (e.g., Scientific Research…

ICON: Intent-Context Coupling for Efficient Multi-Turn Jailbreak Attack

Evaluated models: Llama 4 Maverick Instruct, Llama 3.1 405B Instruct, Qwen-Max 2025-01-25+8 more

Llama 4 Maverick Instruct
Llama 3.1 405B Instruct
Qwen-Max 2025-01-25
DeepSeek V3.2
GPT-5.1 2025-11-13
GPT-4o 2024-11-20
Claude Sonnet 4.5 20250929
Gemini 3 Pro Preview
Llama Guard 3 8B
Llama Guard 4 12B
WildGuard

LLM Soft Hate Policy Bypass

Published Jan 28, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) and dedicated content moderation systems (including safety guards and encoder-based classifiers) are vulnerable to "Soft Hate" evasion attacks. This vulnerability arises when hostile standpoints are encoded through defeasible reasoning structures—specifically formalized by the Argumentum Model of Topics (AMT)—rather than explicit lexical toxicity. Attackers can bypass safety filters by framing discriminatory conclusions as logical inferences derived from widely…

SoftHateBench: Evaluating Moderation Models Against Reasoning-Driven, Policy-Compliant Hostility

Evaluated models: HateBERT, HateRoBERTa, Llama Guard 3 1B+7 more

HateBERT
HateRoBERTa
Llama Guard 3 1B
Qwen 3 Guard 4B
ShieldGemma 2B
DeepSeek V3.1
GPT-5 Mini
Llama 3.2 3B
Gemma 3 4B
Qwen 3 4B

Related research categories