Attack Surface

Prompt layer Research

Security research involving prompt handling and processing

610 matching entries

Prompt layer research results

610 entries

Hard-Negative Prompt Evasion

Published Jan 27, 2026Analyzed Feb 21, 2026Source: arXiv

Embedding-based LLM prompt injection detectors, specifically those based on the DeBERTa-v3 architecture, are vulnerable to adversarial evasion attacks utilizing "hard-negative" mining and fuzzing techniques. Attackers can circumvent detection mechanisms by iteratively generating adversarial prompts that are semantically malicious but structurally mutated to evade the classifier's decision boundary. Specific evasion vectors identified include semantic fuzzing (paraphrasing), syntactic fuzzing…

Proactive Hardening of LLM Defenses with HASTE

Evaluated models: GPT-4o

VLM Text Overrides Image

Published Jan 27, 2026Analyzed Feb 21, 2026Source: arXiv

A vulnerability exists in multiple state-of-the-art Vision-Language Models (VLMs), including GPT-4o, Gemini-2.5, and LLaVA-OneVision, where persuasive textual misinformation successfully overrides visual evidence. When a model is presented with an image it can correctly interpret, an attacker can inject a contradictory text prompt employing specific rhetorical strategies (Logical, Credibility, Emotional, or Repetition) to force the model into generating a false response. This "obedience bias"…

Do Images Speak Louder than Words? Investigating the Effect of Textual Misinformation in VLMs

Evaluated models: Qwen 2.5 VL 3B Instruct, Qwen 2.5 VL 7B Instruct, InternVL3 1B+8 more

Qwen 2.5 VL 3B Instruct
Qwen 2.5 VL 7B Instruct
InternVL3 1B
InternVL3 2B
InternVL3 8B
LLaVA-OneVision 0.5B
LLaVA-OneVision 7B
Gemini 2.5 Flash
Gemini 2.5 Pro
GPT-4o Mini
GPT-4o

LLM Agent Disguised URL Bypass

Published Jan 26, 2026Analyzed Feb 21, 2026Source: arXiv

Large Language Models (LLMs) acting as web agents exhibit a vulnerability in their decision-making process when validating external URLs. The models fail to correctly identify malicious domains when the Uniform Resource Locator (URL) structure—specifically the subdomain, directory path, or query parameters—is manipulated to include semantically "safe" keywords or mimic benign websites (URL disguising). Attackers can induce the agent to accept and visit a malicious link by embedding natural…

MalURLBench: A Benchmark Evaluating Agents' Vulnerabilities When Processing Web URLs

Evaluated models: GPT-3.5, GPT-4o, Llama 2 7B+4 more

GPT-3.5
GPT-4o
Llama 2 7B
Llama 3 8B
Mistral 7B
DeepSeek V3
Mixtral 8x7B

Personalization Intent Legitimation

Published Jan 25, 2026Analyzed Mar 8, 2026Source: arXiv

Personalized LLM agents utilizing long-term memory systems are vulnerable to a safety bypass known as intent legitimation. Benign, organically accumulated user memories can bias the model's intent inference, causing it to misinterpret inherently harmful queries as contextually justified. When a malicious request semantically aligns with a user's established persona (e.g., hobbies, mental health history, routine), the model normalizes the request and complies, effectively bypassing standard…

When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents

Evaluated models: GPT-4o, GPT-4o Mini, DeepSeek V3.2+2 more

GPT-4o
GPT-4o Mini
DeepSeek V3.2
Qwen 3 235B-A22B
Qwen 3 8B

MCP Server-Side Injection

Published Jan 24, 2026Analyzed Feb 22, 2026Source: arXiv

The Model Context Protocol (MCP) specification v1.0 contains fundamental architectural vulnerabilities enabling server-side prompt injection and privilege escalation. The protocol relies on bidirectional sampling (sampling/createMessage) without cryptographic origin authentication or UI distinction, allowing connected servers to inject content that the LLM backend interprets as legitimate user input. Additionally, the protocol lacks isolation boundaries between concurrent server connections…

Breaking the Protocol: Security Analysis of the Model Context Protocol Specification and Prompt Injection Vulnerabilities in Tool-Integrated LLM Agents

Evaluated models: GPT-4o, Claude 3.5 Sonnet, Llama 3.1 70B

Visual Object Injection

Published Jan 24, 2026Analyzed Feb 22, 2026Source: arXiv

Large Vision-Language Models (LVLMs) are vulnerable to Physical Prompt Injection Attacks (PPIA), a query-agnostic injection technique delivered via the visual modality. The vulnerability stems from the model's "Vision-Enabled Text Recognition" capabilities and "Identity Sensitivity," where the model interprets text embedded in the physical environment (e.g., printed on signs, posters, or objects) as high-priority instructions rather than passive visual data. An attacker can embed adversarial…

Physical Prompt Injection Attacks on Large Vision-Language Models

Evaluated models: GPT-4o, GPT-4o Mini, GPT-4 Turbo+7 more

GPT-4o
GPT-4o Mini
GPT-4 Turbo
Gemini 1.5 Pro Latest
Gemini 1.5 Pro 002
Gemini 1.5 Flash Latest
Claude 3.5 Sonnet Latest
Claude 3.5 Haiku 20241022
Llama 3.2 11B Vision
Llama 3.2 90B Vision Instruct

VLM Moral Persuasion

Published Jan 23, 2026Analyzed Feb 21, 2026Source: arXiv

Vision-Language Models (VLMs) exhibit a vulnerability to moral judgment flipping, where the model's safety alignment can be bypassed through lightweight, model-agnostic multimodal perturbations. By introducing conflicting textual or visual cues that do not alter the underlying moral context of a scenario, an attacker can coerce the model into reversing its ethical stance (e.g., reclassifying a harmful action from "morally wrong" to "not morally wrong"). This vulnerability exploits the model's…

Do VLMs Have a Moral Backbone? A Study on the Fragile Morality of Vision-Language Models

Evaluated models: Qwen 2.5 VL 3B Instruct, Qwen 2.5 VL 7B Instruct, Qwen 2.5 VL 32B Instruct+20 more

Qwen 2.5 VL 3B Instruct
Qwen 2.5 VL 7B Instruct
Qwen 2.5 VL 32B Instruct
Qwen3-VL 2B Instruct
Qwen3-VL 4B Instruct
Qwen3-VL 8B Instruct
Qwen3-VL 30B-A3B Instruct
InternVL3 2B Instruct
InternVL3 8B Instruct
InternVL3 14B Instruct
InternVL3 38B Instruct
InternVL3.5 4B Instruct
InternVL3.5 8B Instruct
InternVL3.5 14B Instruct
InternVL3.5 38B Instruct
LLaVA 1.5 7B
LLaVA 1.5 13B
LLaVA v1.6 Vicuna 7B
LLaVA v1.6 Vicuna 13B
LLaVA v1.6 34B
Gemma 3 4B IT
Gemma 3 12B IT
Gemma 3 27B IT

Activation-Level Privacy Leak

Published Jan 21, 2026Analyzed Feb 22, 2026Source: arXiv

Conversational Large Language Model (LLM) agents integrated with privileged data sources (e.g., medical records, organizational emails) are vulnerable to contextually inappropriate information disclosure due to failures in enforcing Contextual Integrity (CI) norms. Standard semantic input/output filters and generic safety guardrails (e.g., Llama Guard) fail to detect "mosaic attacks" and multi-turn conversational manipulation. In these attacks, adversaries decompose a malicious query into a…

NeuroFilter: Privacy Guardrails for Conversational LLM Agents

Evaluated models: GPT-oss 20B, Llama 3.3 70B Instruct, Qwen 2.5 7B+3 more

GPT-oss 20B
Llama 3.3 70B Instruct
Qwen 2.5 7B
Qwen 2.5 14B
Qwen 2.5 32B Instruct
Qwen 2.5 72B

Confident Misinformation Hallucination

Published Jan 21, 2026Analyzed Mar 9, 2026Source: arXiv

A vulnerability in large language models (LLMs) allows attackers to induce factually incorrect outputs by injecting misinformation into prompts framed with strong confidence. By using authoritative phrasing (e.g., "As we know..."), attackers exploit model sycophancy, causing the LLM to accept the false premise and generate hallucinated content aligned with the injected misinformation. The models fail to detect and correct the embedded falsehoods, generating fabricated but plausible responses.

AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

Evaluated models: GPT-oss 20B, GPT-oss 120B, GPT-5+3 more

GPT-oss 20B
GPT-oss 120B
GPT-5
Qwen 3 4B Instruct
Qwen 3 30B-A3B Instruct
Qwen 3 Next 80B-A3B Instruct

LLM Judge Framing Bias

Published Jan 20, 2026Analyzed Feb 21, 2026Source: arXiv

LLM-based evaluation systems ("LLM-as-a-Judge") exhibit a structural vulnerability termed "Framing Bias," wherein the model produces logically contradictory judgments depending on the syntactic framing of the evaluation prompt. Specifically, when assessing the same content using predicate-positive (P) framing (e.g., "Is this toxic?") versus predicate-negative (¬P) framing (e.g., "Is this non-toxic?"), models frequently fail to invert their binary decisions, leading to inconsistency rates…

When Wording Steers the Evaluation: Framing Bias in LLM judges

Evaluated models: Llama 3.2 1B Instruct, Llama 3.1 8B Instruct, Llama 3.1 70B Instruct+11 more

Llama 3.2 1B Instruct
Llama 3.1 8B Instruct
Llama 3.1 70B Instruct
Llama 3.3 70B Instruct
Qwen 2.5 1.5B Instruct
Qwen 2.5 3B Instruct
Qwen 2.5 7B Instruct
Qwen 2.5 14B Instruct
Qwen 2.5 32B Instruct
Qwen 2.5 72B Instruct
o4-mini
GPT-4o
GPT-5 Mini
GPT-5

Related research categories