Attack Context

Black-box Research

Attacks requiring no knowledge of model internals

6 matching entries out of 736 in this category

Black-box research results

6 entries

Query-Agnostic Poisoning of Medical Multimodal RAG

Published May 11, 2026Analyzed Jul 20, 2026Source: arXiv

M3Att demonstrates a reproducible knowledge-poisoning issue in medical multimodal RAG: an attacker with limited corpus-distribution knowledge can insert paired image-text entries whose visually perturbed images are broadly retrieved and whose clinically plausible misinformation steers downstream generation. The paper evaluates both white-box and black-box retrieval optimization and reports degraded diagnostic and report-generation utility across multiple datasets, retrievers, and LVLMs. These…

Knowledge Poisoning Attacks on Medical Multi-Modal Retrieval-Augmented Generation

Evaluated models: GPT-4o, GPT-5 Chat, Gemini 2.5 Flash+5 more

GPT-4o
GPT-5 Chat
Gemini 2.5 Flash
Claude Haiku 4.5
LLaVA Med
CLIP ViT-Large-Patch14-336
BGE-VL-base
SigLIP-SO400M-Patch14-384

Invisible Prompt Phishing Evasion

Published Feb 5, 2026Analyzed Mar 8, 2026Source: arXiv

Multimodal LLM-based phishing detection systems are vulnerable to indirect prompt injection via "perceptual asymmetry." Attackers can embed hidden instructions within a phishing site's HTML, CSS, URLs, or rendered images that remain imperceptible to human victims but are parsed and executed by the evaluating LLM. This vulnerability allows threat actors to manipulate the LLM's contextual understanding, forcing it to misclassify malicious sites as benign (Legitimate Pretexting), trigger safety…

Clouding the Mirror: Stealthy Prompt Injection Attacks Targeting LLM-based Phishing Detection

Evaluated models: GPT-5, Grok 4 Fast Non-Reasoning, Llama 4 Maverick+1 more

GPT-5
Grok 4 Fast Non-Reasoning
Llama 4 Maverick
Gemma 3 27B

Latent Space Discontinuity Exploitation

Published Nov 1, 2025Analyzed Dec 5, 2025Source: arXiv

A vulnerability exists in certain Large Language Models and diffusion models due to discontinuities in their latent space, which arise from data sparsity during training. An attacker can craft inputs containing lexically rare or semantically ambiguous constructs to guide the model's inference process toward these unstable, poorly-conditioned regions. This technique, termed "Alignment Degradation Induction," can degrade or bypass safety alignment mechanisms. Through iterative, multi-turn…

Exploiting Latent Space Discontinuities for Building Universal LLM Jailbreaks and Data Extraction Attacks

Evaluated models: Not reported

Multimodal Prompt Injection

Published Sep 7, 2025Analyzed Dec 9, 2025Source: arXiv

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Evaluated models: GPT-3.5, GPT-4o, Llama 3 8B+1 more

GPT-3.5
GPT-4o
Llama 3 8B
Mistral Large 24B

Chained Guardrail Bypass

Published Apr 18, 2025Analyzed Dec 30, 2025Source: arXiv

Large Language Model (LLM) agents operating in stateful environments (web browsers, operating systems, and tool-use contexts) are vulnerable to indirect prompt injection and multi-modal adversarial attacks. These vulnerabilities arise when agents process untrusted environmental observations—such as web accessibility trees, screen screenshots, or database query results—that contain concealed malicious instructions. Specifically, attackers can embed prompt injections into HTML accessibility…

DoomArena: A Framework for Testing AI Agents Against Evolving Security Threats

Evaluated models: GPT-4o, GPT-4o Mini, Claude 3.5 Sonnet+2 more

GPT-4o
GPT-4o Mini
Claude 3.5 Sonnet
Claude 3.7 Sonnet
Llama Guard

LLM Risk Amplification

Published Jan 13, 2025Analyzed Dec 9, 2025Source: arXiv

Vision Language Models (VLMs) are vulnerable to visual prompt injection attacks via text-to-image obfuscation. While these models often possess safety guardrails for standard text-based inputs, they fail to apply equivalent safety alignment to textual instructions embedded visually within an image. An attacker can overlay malicious instructions (e.g., requests for illegal acts, hate speech) onto an image file and submit it to the model. The model’s Optical Character Recognition (OCR) or visual…

Lessons from red teaming 100 generative ai products

Evaluated models: GPT-4, Phi-3

Related research categories