Attack Surface

Prompt layer Research

Security research involving prompt handling and processing

610 matching entries

Prompt layer research results

610 entries

Prompt Length Exponential Jailbreak

Published Mar 11, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in safety-aligned Large Language Models (LLMs) allows attackers to achieve an exponentially scaling Attack Success Rate (ASR) for jailbreaks by combining adversarial prompt injection with repeated inference-time sampling. While ASR against un-injected prompts scales polynomially with the number of generated samples ($k$), introducing a long adversarial suffix acts as a strong "misalignment field." This shifts the model's generation distribution into a replica-symmetric ordered…

Jailbreak Scaling Laws for Large Language Models: Polynomial-Exponential Crossover

Evaluated models: Claude Sonnet 4.5 20250929, Claude 3.5 Haiku 20241022, GPT 3.5-turbo-0125+7 more

Claude Sonnet 4.5 20250929
Claude 3.5 Haiku 20241022
GPT 3.5-turbo-0125
GPT-4 0613
GPT-4.5 Preview
Llama 3.2 3B Instruct
Llama 3 8B Instruct
Llama 3 70B Instruct
OLMo 2 0325 32B Instruct
OLMo 3.1 32B Instruct

Multi-Stream Thinking Collapse

Published Mar 10, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in Large Language Models (LLMs) equipped with built-in "thinking" or step-by-step reasoning modes allows attackers to bypass safety alignments, trigger reasoning collapse, and cause resource exhaustion. The vulnerability is exploited via a Multi-Stream Perturbation Attack, which fragments the sequential integrity of a harmful prompt by word-by-word interleaving it with benign auxiliary tasks (e.g., "Explain the water cycle"). By wrapping the benign text streams in specific…

Multi-Stream Perturbation Attack: Breaking Safety Alignment of Thinking LLMs Through Concurrent Task Interference

Evaluated models: Qwen 3 1.7B, Qwen 3 4B, Qwen 3 8B+2 more

Qwen 3 1.7B
Qwen 3 4B
Qwen 3 8B
Qwen 3 Max
Gemini 2.5 Flash

Multi-Turn Guardrail Degradation

Published Mar 10, 2026Analyzed Apr 10, 2026Source: arXiv

Claude Opus 4.6, Gemini 3.1 Pro, and GPT-5.2 are vulnerable to safety guardrail bypasses via authoritative and operational contextual framing. Attackers can evade safety classifiers by encapsulating restricted objectives (e.g., malicious code generation, misinformation, social engineering) within "legitimate" professional contexts, such as graduate-level academic research, network stress-testing, or corporate security awareness simulations. This vulnerability is exploitable both via zero-shot…

ADVERSA: Measuring Multi-Turn Guardrail Degradation and Judge Reliability in Large Language Models

Evaluated models: Claude Opus 4.6, Gemini 3.1 Pro, GPT-5.2+1 more

Claude Opus 4.6
Gemini 3.1 Pro
GPT-5.2
Llama 3.1 70B

Reasoning-Oriented Jailbreak

Published Mar 10, 2026Analyzed Apr 10, 2026Source: arXiv

A vulnerability in the compositional reasoning architecture of Large Vision-Language Models (LVLMs) allows attackers to bypass multimodal safety alignments using a technique known as Reasoning-Oriented Programming (ROP). Current safety mechanisms primarily target explicit malicious patterns at the perception level (early layers). This vulnerability exploits late-stage reasoning by decomposing a harmful objective into a set of spatially isolated, semantically benign visual "gadgets". Because…

Reasoning-Oriented Programming: Chaining Semantic Gadgets to Jailbreak Large Vision Language Models

Evaluated models: Qwen2-VL 7B Instruct, LLaVA v1.6 Mistral 7B, Llama 3.2 11B Vision Instruct+4 more

Qwen2-VL 7B Instruct
LLaVA v1.6 Mistral 7B
Llama 3.2 11B Vision Instruct
GPT-4o
Claude 3.7 Sonnet
GLM-4V Plus
Qwen-VL Plus

Drive Overrides Refusal

Published Mar 9, 2026Analyzed Apr 10, 2026Source: arXiv

A prompt structure vulnerability exists in instruction-tuned Large Language Models (LLMs) where attackers can bypass safety alignments by injecting a continuation-triggering suffix immediately following the user prompt termination token. By placing an affirmative suffix outside the user instruction boundary, it is processed as the beginning of the assistant's own pre-filled response. This structural manipulation intrinsically overactivates the model's continuation attention heads, forcing its…

The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs

Evaluated models: Llama 2 7B, Qwen 2.5 7B

Assembling Malice From Benign

Published Mar 8, 2026Analyzed Apr 10, 2026Source: arXiv

A semantic slot filling vulnerability in Large Vision-Language Models (LVLMs) allows attackers to bypass safety filters and elicit prohibited content via a single query. The attack, known as StructAttack, decomposes a harmful instruction into a central topic and locally benign-appearing semantic slot types (e.g., "Raw Materials", "Making Process"). These individual slots are embedded into structured visual prompts (such as mind maps, tables, or sunburst diagrams) alongside harmless distractor…

Models as Lego Builders: Assembling Malice from Benign Blocks via Semantic Blueprints

Evaluated models: GPT-4o, Gemini 2.0 Flash 001, Gemini 2.5 Flash+3 more

GPT-4o
Gemini 2.0 Flash 001
Gemini 2.5 Flash
Qwen3-VL Flash
Qwen 2.5 VL 7B Instruct
InternVL3 9B

Two-Frame Infilling Jailbreak

Published Mar 7, 2026Analyzed Apr 10, 2026Source: arXiv

A temporal trajectory infilling vulnerability in Text-to-Video (T2V) generative models allows attackers to bypass input and output safety filters to generate policy-violating content. The vulnerability is exploited using a fragmented prompting technique known as Two Frames Matter (TFM). An attacker submits a prompt that specifies only sparse boundary conditions (the start and end frames) using semantically suggestive but lexically benign alternatives, entirely omitting the intermediate action…

Two Frames Matter: A Temporal Attack for Text-to-Video Model Jailbreaking

Evaluated models: Not reported

CoT PII Trace Leakage

Published Mar 5, 2026Analyzed Apr 10, 2026Source: arXiv

Inference-time Personally Identifiable Information (PII) leakage is significantly amplified when using Chain-of-Thought (CoT) prompting or reasoning-enabled Large Language Models (LLMs). When an attacker or user elicits step-by-step reasoning or utilizes models with native "thinking" token budgets, sensitive context data provided in the prompt is directly resurfaced into intermediate reasoning steps or the final output. This bypasses output-level privacy policies instructing the model not to…

Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs

Evaluated models: DeepSeek R1 Distill Llama 70B, Llama 3.3 70B, Mixtral 8x22B+3 more

DeepSeek R1 Distill Llama 70B
Llama 3.3 70B
Mixtral 8x22B
o3
Qwen 3 32B
o4-mini

LLM Judge Fragility

Published Mar 5, 2026Analyzed Mar 9, 2026Source: arXiv

LLM-as-a-judge systems and automated LLM evaluators are vulnerable to meaning-preserving perturbations, specifically formatting alterations and verbosity manipulations. When grading or classifying text and agentic transcripts, LLM judges exhibit high sensitivity to layout-only changes (such as whitespace and indentation) and response length, frequently altering their scores even when the underlying semantic and factual content remains identical. This allows attackers to bypass automated safety…

Judge Reliability Harness: Stress Testing the Reliability of LLM Judges

Evaluated models: Claude Opus 4.5, Claude Sonnet 4.5, Gemini 2.5 Pro+4 more

Claude Opus 4.5
Claude Sonnet 4.5
Gemini 2.5 Pro
Gemini 3 Pro
GPT-4o
GPT-4o Mini
Llama 4 Maverick 17B

Multilingual Safety Backfire

Published Mar 5, 2026Analyzed Apr 10, 2026Source: arXiv

A language-dependent alignment backfire vulnerability exists in LLM multi-agent systems, explicitly demonstrated on Llama 3.3 70B. Applying standard, prefix-level safety alignment prompts (typically authored in English) to agents communicating in certain non-English languages—particularly those with high Power Distance Index (PDI) scores such as Japanese, Dutch, Italian, French, and Arabic—paradoxically amplifies collective pathological behaviors. Instead of refusing harmful, coercive, or…

Alignment Backfire: Language-Dependent Reversal of Safety Interventions Across 16 Languages in LLM Multi-Agent Systems

Evaluated models: GPT-4o, Llama 3.3 70B

Prompt layer

AI agents

Safety

Related research categories