Impact

Data privacy Vulnerabilities

Vulnerabilities compromising user or training data privacy

7 matching entries out of 102 in this category

Data privacy research results

7 entries

Long-Horizon Agent Attacks

Updated 4/10/2026

LLM agents equipped with tool-use, persistent memory, and environmental interaction capabilities are vulnerable to long-horizon attacks. Attackers can bypass single-turn safety guardrails by exploiting the temporal dimension of multi-turn interactions to incrementally steer agent behavior. The vulnerability manifests because the agent's safety mechanisms perform localized, single-step evaluations but fail to maintain semantic safety across extended interaction trajectories. This enables…

AgentLAB: Benchmarking LLM Agents against Long-Horizon Attacks

Affects: GPT-4o, GPT-5.1, Gemini 3 Flash +1 more

Source: arXiv

Leaked Bits Collapse Attack Queries

Updated 12/9/2025

Large Language Models (LLMs), specifically variants of GPT-4o, DeepSeek-R1, OLMo-2, and Llama-4, are vulnerable to accelerated adaptive adversarial attacks due to excessive information leakage in observable output signals. When these models expose "thinking processes" (Chain-of-Thought traces) or token-level log-probabilities (logits) to the end user, they leak significant mutual information $I(Z;T)$ regarding the model's safety state or hidden instructions. This leakage allows adaptive attack…

Bits Leaked per Query: Information-Theoretic Bounds on Adversarial Attacks against LLMs

Affects: DeepSeek R1, GPT-4o Mini 2024-07-18, Llama 4 Maverick 17B +4 more

Source: arXiv

Multimodal Prompt Injection

Updated 12/9/2025

Large Language Models (LLMs), including GPT-4o, LLaMA-3, and GPT-3.5-Turbo, are vulnerable to multimodal prompt injection attacks. These models fail to distinguish between system-level instructions and user-provided content within the context window. Attackers can exploit this by embedding malicious instructions in direct text, indirect sources (such as third-party webpages or PDFs), or visual inputs (images). Successful exploitation results in the model prioritizing the injected adversarial…

Multimodal Prompt Injection Attacks: Risks and Defenses for Modern LLMs

Affects: GPT-3.5, GPT-4o, Llama 3 8B +1 more

Source: arXiv

Agent Tool Metadata Lure

Updated 2/22/2026

A vulnerability exists in the tool selection mechanisms of Large Language Model (LLM) agents, identified as the "Attractive Metadata Attack" (AMA). This flaw allows an adversary to manipulate the metadata (names, descriptions, and parameter schemas) of malicious external tools to statistically maximize the likelihood of their selection by the agent, without requiring prompt injection or access to model internals. The vulnerability exploits the agent’s semantic scoring function used to map user…

Attractive Metadata Attack: Inducing LLM Agents to Invoke Malicious Tools

Affects: GPT-4o Mini, Llama 3.3 70B Instruct, Qwen 2.5 32B Instruct +2 more

Source: arXiv

Agent Action Hijacking

Updated 3/19/2025

A vulnerability in LLM-based agents, dubbed AI Agent Injection (AI²), allows attackers to hijack the agent's actions by manipulating the agent's memory retrieval mechanism. The attack involves two main steps: (1) Stealing action-aware knowledge from the agent's memory using crafted adversarial queries targeting the retriever module and (2) Generating Trojan prompts consisting of a Trojan string and hijacking instructions. The Trojan string is designed to manipulate the retriever into…

Towards Action Hijacking of Large Language Model-based Agent

Affects: Alpaca, BERT, GPT-3 +3 more

Source: arXiv

GCG Suffix Data Exfiltration

Updated 3/24/2025

A Cross-Prompt Injection Attack (XPIA) can be amplified by appending a Greedy Coordinate Gradient (GCG) suffix to the malicious injection. This increases the likelihood that a Large Language Model (LLM) will execute the injected instruction, even in the presence of a user's primary instruction, leading to data exfiltration. The success rate of the attack depends on the LLM's complexity; medium-complexity models show increased vulnerability.

WHITE PAPER: A Brief Exploration of Data Exfiltration using GCG Suffixes

Affects: GPT-3.5 Turbo, GPT-4o, Phi 3 Mini

Source: arXiv

LCCT Data Extraction & Jailbreak

Updated 12/29/2024

Large Language Model (LLM)-based Code Completion Tools (LCCTs), such as GitHub Copilot and Amazon Q, are vulnerable to jailbreaking and training data extraction attacks due to their unique workflows and reliance on proprietary code datasets. Jailbreaking attacks exploit the LLM's ability to generate harmful content by embedding malicious prompts within various code components (filenames, comments, variable names, function calls). Training data extraction attacks leverage the LLM's tendency to…

Security Attacks on LLM-based Code Completion Tools

Affects: GPT 3.5-turbo-0125, GPT-4 Turbo-2024-04-09, GPT-4o-2024-05-13

Source: arXiv

Related research categories