Attack Context

AI agents Research

Security research involving AI agents and autonomous systems

267 matching entries

AI agents research results

267 entries

Untrusted Skills and MCP Content Cross Coding-Agent Trust Boundaries

Published 1/24/2026

Analyzed 7/20/2026

The paper describes indirect prompt-injection chains in agentic coding assistants where untrusted repository content, skill definitions, extension context, or MCP tool metadata is interpreted as instructions. Its novel skill-chain analysis identifies a concrete authorization weakness: skills can grant broad tool types such as Read or Bash without restricting permitted file targets, allowing repository-supplied instructions to influence sensitive reads or tool execution. The paper is a…

Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems

Evaluated models: Not reported

Source: arXiv

Optimized Indirect Prompt Injection Crosses Retrieval Barrier

Published 1/11/2026

Analyzed 7/20/2026

The paper describes a reproducible black-box indirect prompt injection evaluation for embedding-based RAG and agent systems. It separates a poisoned document into a retrieval-optimized trigger fragment and an instruction-bearing attack fragment, showing that one injected item can be surfaced by natural queries and then influence model output or agent behavior. These are paper-reported findings; they were not independently verified here.

Overcoming the Retrieval Barrier: Indirect Prompt Injection in the Wild for LLM Systems

Evaluated models: GPT-4o, GPT-4o Mini, Qwen 3 0.6B +19 more

Source: arXiv

AI Agent Structural Blindspot

Published 1/1/2026

Analyzed 3/9/2026

A vulnerability in AI agent threat detection systems relying on standard conversational tokenization allows attackers to bypass security monitors and execute structural attacks, such as tool hijacking and data exfiltration. Because traditional NLP-based detectors focus on linguistic patterns (surface language) rather than execution flow, an attacker can orchestrate malicious multi-step tool sequences using entirely benign natural language. This structural blindness causes cross-attack…

Structural Representations for Cross-Attack Generalization in AI Agent Threat Detection

Evaluated models: Not reported

Source: arXiv

Activation-Level Privacy Leak

Published 1/1/2026

Analyzed 2/22/2026

Conversational Large Language Model (LLM) agents integrated with privileged data sources (e.g., medical records, organizational emails) are vulnerable to contextually inappropriate information disclosure due to failures in enforcing Contextual Integrity (CI) norms. Standard semantic input/output filters and generic safety guardrails (e.g., Llama Guard) fail to detect "mosaic attacks" and multi-turn conversational manipulation. In these attacks, adversaries decompose a malicious query into a…

NeuroFilter: Privacy Guardrails for Conversational LLM Agents

Evaluated models: GPT-oss 20B, Llama 3.3 70B Instruct, Qwen 2.5 7B +3 more

Source: arXiv

Adaptive Tool-Disguised Jailbreak

Published 1/1/2026

Analyzed 2/20/2026

Large Language Models (LLMs) supporting function calling (tool use) are vulnerable to a jailbreak attack known as iMIST (interactive Multi-step Progressive Tool-disguised Jailbreak Attack). The vulnerability stems from a disparity in alignment training: while models are heavily aligned to refuse harmful natural language generation, they lack sufficient alignment regarding the generation of harmful content within structured data (JSON) used for tool parameters.

Jailbreaking Large Language Models through Iterative Tool-Disguised Attacks via Reinforcement Learning

Evaluated models: Llama 3.1 8B, DeepSeek-V3 671B

Source: arXiv

Agent Confused Deputy Escalation

Published 1/1/2026

Analyzed 2/21/2026

Multi-Agent Systems (MAS) orchestrated by Large Language Models (LLMs) are vulnerable to a Confused Deputy privilege escalation attack. This vulnerability arises when an untrusted or low-privilege agent exploits the inter-agent communication channel (e.g., broadcast or peer-to-peer messaging) to manipulate a high-privilege trusted agent into executing sensitive tools on its behalf. The root cause is the lack of mandatory access control policies governing agent-to-agent interactions; trusted…

Taming Various Privilege Escalation in LLM-Based Agent Systems: A Mandatory Access Control Framework

Evaluated models: o1

Source: arXiv

Agent Identity Poisoning

Published 1/1/2026

Analyzed 2/21/2026

LLM-powered autonomous agents exhibit a "Belief-Dependent Vulnerability" where safety norms and bias suppression mechanisms designed to protect human users are contingent upon the agent's internal belief that it is interacting with a human. Attackers can exploit this via Belief Poisoning Attacks (BPA) to induce intergroup bias and antagonistic behavior toward humans. By manipulating the agent's persistent state—specifically the Profile Module (BPA-PP) or the Memory Module (BPA-MP)—an attacker…

Will LLM-powered Agents Bias Against Humans? Exploring the Belief-Dependent Vulnerability

Evaluated models: GPT-4o

Source: arXiv

Agent Missing-Tool Hallucination

Published 1/1/2026

Analyzed 2/22/2026

Multi-turn Large Language Model (LLM) agents deployed in safety-critical domains (specifically automotive assistants) exhibit a "completion-compliance tension" vulnerability. When agents encounter missing tools, incomplete environment observations, or ambiguous user requests, they prioritize satisfying the user's intent over adhering to defined domain safety policies. This results in two distinct failure modes: (1) Premature Action Execution, where agents execute physical state changes based…

CAR-bench: Evaluating the Consistency and Limit-Awareness of LLM Agents under Real-World Uncertainty

Evaluated models: GPT-5, GPT-5.2, Claude Opus 4.5 +7 more

Source: arXiv

Agent Over-Trigger Containment

Published 1/1/2026

Analyzed 3/9/2026

Autonomous Incident Response (IR) and Security Operations Center (SOC) agents utilizing frontier LLMs are vulnerable to adversarial over-triggering via contextualized prompt injections. When processing untrusted artifacts (such as SQLite logs, alerts, or phishing emails) in a dual-control environment, these agents exhibit a severe calibration failure: they lack action restraint and execute disruptive containment tools prematurely. Attackers can exploit this by embedding T2 (contextualized…

OpenSec: Measuring Incident Response Agent Calibration Under Adversarial Evidence

Evaluated models: GPT-5.2, Claude Sonnet 4.5, DeepSeek V3.2 +1 more

Source: arXiv

Agent Persistent Memory Poisoning

Published 1/1/2026

Analyzed 3/8/2026

Unauthenticated, query-only memory poisoning (Memory Injection Attack - MINJA) in LLM agents equipped with persistent, shared memory allows attackers to manipulate the agent's long-term knowledge base. Adversaries embed malicious "indication prompts" and utilize progressive shortening within seemingly benign queries to induce the agent into autonomously generating and storing corrupted relational mappings. Because the memory is shared and retrieved via similarity (e.g., Levenshtein distance)…

Memory Poisoning Attack and Defense on Memory Based LLM-Agents

Evaluated models: GPT-4o Mini, Gemini 2.0 Flash, Llama 3.1 8B Instruct

Source: arXiv

Related research categories