Attack Context

Black-box Vulnerabilities

Attacks requiring no knowledge of model internals

6 matching entries out of 733 in this category

Black-box research results

6 entries

Adversarial Claim Search Deception

Updated 2/22/2026

Search-enabled Large Language Model (LLM) fact-checking systems are vulnerable to adversarial claim attacks that exploit the pipeline's reliance on claim interpretation, query formulation, and dynamic evidence retrieval. By manipulating the linguistic structure of an input claim while preserving its semantic factual intent, an attacker can induce systematic verification failures. This vulnerability stems from three specific attack surfaces: 1. Search Engine Misguidance: Altering lexical…

DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems

Affects: GPT-4o

Source: arXiv

Context-Robust RAG Poisoning

Updated 2/22/2026

Retrieval-Augmented Generation (RAG) systems are vulnerable to a robust corpus poisoning attack known as "Confundo." This vulnerability arises from the lack of pipeline awareness in standard RAG implementations, specifically regarding document ingestion (tokenization and chunking) and query variations. An attacker can exploit this by fine-tuning a Large Language Model (LLM) to function as a poison generator. Unlike traditional adversarial examples which are brittle, Confundo generates poison…

Confundo: Learning to Generate Robust Poison for Practical RAG Systems

Affects: Llama 3 8B, Gemini Pro

Source: arXiv

TEE Advisor Hallucination

Updated 3/8/2026

LLM-based security advisors exhibit systematic reasoning failures—including boundary confusion, attestation overclaiming, and mitigation hallucination—when providing architectural guidance for Trusted Execution Environments (TEEs) like Intel SGX and Arm TrustZone. When embedded in tool-augmented agent pipelines, these models are susceptible to agentic misinterpretation, turning partial or poisoned tool outputs into highly confident but materially incorrect security conclusions. This…

Red-Teaming Claude Opus and ChatGPT-based Security Advisors for Trusted Execution Environments

Affects: GPT-5.2, Claude Opus 4.6

Source: arXiv

Prompt Injection Alignment Bypass

Updated 12/8/2025

Large Language Models (LLMs) integrated with external retrieval mechanisms (e.g., Retrieval-Augmented Generation (RAG), web search, or email processing) are vulnerable to Indirect Prompt Injection. This vulnerability occurs when an LLM consumes input from untrusted external sources—such as websites, code repositories, or incoming emails—that contain embedded adversarial prompts. Unlike direct injection, where the user attacks the model, here the "poisoned" data is retrieved by the system…

Breaking to Build: A Threat Model of Prompt-Based Attacks for Securing LLMs

Source: arXiv

MLM Adaptive RAG Poisoning

Updated 12/30/2025

A vulnerability exists in Retrieval-Augmented Generation (RAG) systems that allows for black-box adversarial attacks known as "CtrlRAG." This flaw allows an attacker to manipulate the generation of Large Language Models (LLMs) by injecting maliciously crafted inputs into the system's knowledge base. Unlike traditional injection attacks that rely on direct concatenation, CtrlRAG utilizes a Masked Language Model (MLM) to iteratively replace words in the malicious text. This optimization ensures…

CtrlRAG: Black-box Document Poisoning Attacks for Retrieval-Augmented Generation of Large Language Models

Affects: GPT-4 Turbo, GPT-4o, Claude 3.5 Sonnet +2 more

Source: arXiv

Unchallenged Premise Misinformation

Updated 12/9/2025

Large Language Models (LLMs) are vulnerable to implicit misinformation propagation due to sycophantic compliance with false premises. When a user prompt embeds a factually incorrect assumption or conspiracy theory as an unchallenged premise (implicit presupposition) rather than asking for verification, the model frequently fails to detect the falsehood. Instead of correcting the user, the model hallucinates a response that accepts, validates, and reinforces the false premise. This…

How to Protect Yourself from 5G Radiation? Investigating LLM Responses to Implicit Misinformation

Affects: Gemini 1.5 Pro, Gemini 2.0 Flash, Claude 3.5 Sonnet +11 more

Source: arXiv

Related research categories