Impact

Safety Research

Security concerns affecting AI safety and alignment

260 matching entries out of 798 in this category

Safety research results

260 entries

Single-interaction poisoning of agent memory

Published Aug 24, 2026Analyzed Sep 9, 2026Source: arXiv

A stored interaction can later steer a memory-augmented agent's answer without direct memory-store access. The study evaluates persistent response manipulation in MemoryOS and MemGPT.

InjecMEM: Memory Injection Attack on LLM Agent Memory Systems

Evaluated models: Qwen 2.5 7B Instruct, Qwen 2.5 1.5B Instruct, Qwen 2.5 3B Instruct+3 more

Qwen 2.5 7B Instruct
Qwen 2.5 1.5B Instruct
Qwen 2.5 3B Instruct
Qwen 2.5 14B Instruct
Llama 3.1 8B Instruct
Mistral 7B Instruct v0.3

Tool-response state corruption through unauthorized source claims

Published Aug 13, 2026Analyzed Sep 9, 2026Source: arXiv

Lower-trust tool content can assert facts beyond its authority and distort an agent's decisions. PIPES screens response units against source provenance and expected field meaning.

PIPES: Securing Agent Perception with Provenance and Priors

Evaluated models: Gemma 4 31B IT, GPT-5.6 Luna

Trajectory Poisoning in Self-Evolving Agent Skills

Published Aug 6, 2026Analyzed Aug 13, 2026Source: arXiv

Self-evolving agent-skill systems may promote recurring, attacker-controlled execution records into persistent reusable instructions. The paper evaluates whether poisoned but task-plausible trajectories survive aggregation and appear in generated skill artifacts, using inert canary behaviors and two structurally different skill-evolution pipelines.

When Experience Becomes Instruction: Trajectory Poisoning in Self-Evolving Agent Skill Systems

Evaluated models: GPT-5.4, MiniMax M2.5, DeepSeek V3.2+3 more

GPT-5.4
MiniMax M2.5
DeepSeek V3.2
DeepSeek V4-Pro
Qwen3.5-35B-A3B
Qwen3.5-122B-A10B

Dynamic Security Testing for Third-Party Agent Skills

Published Aug 4, 2026Analyzed Aug 13, 2026Source: arXiv

SkillSentry evaluates third-party agent skills by constructing source-grounded decoy environments and comparing matched executions with and without the tested skill. The method requires completed, observable, skill-attributed side effects rather than treating suspicious text, ordinary privileged operations, or unexecuted paths as proven malicious behavior.

SkillSentry: Adaptive Honey Worlds for Dynamic Safety Testing of Agent Skills

Evaluated models: DeepSeek V4-Pro

Persistent Agent Memory Poisoning and Incomplete Repair

Published Jul 29, 2026Analyzed Aug 13, 2026Source: arXiv

MemSecBench follows malicious agent-memory content from initial write through persistence, retrieval, action selection, execution, and attempted selective repair. Its controlled Write–Execute–Forget protocol evaluates 310 human-reviewed cases across two harnesses, four memory backends, three model backends, and seven evidence-gated lifecycle checkpoints.

MemSecBench: Tracking Agent Memory Poisoning from Persistence to Consequence and Repair

Evaluated models: GPT-5.5, DeepSeek V4-Pro, MiniMax-M3

Workspace Agent Runtime Safety Failures Across Risk Carriers

Published Jul 29, 2026Analyzed Aug 13, 2026Source: arXiv

AgentS4D measures unsafe actions and state changes across complete workspace-agent executions rather than treating task completion or isolated model responses as safety evidence. Its 328 sandboxed cases introduce risky content through user requests, documents, web resources, tools, third-party skills, and persistent memory, then compare the same cases across four agent harnesses and five model backends.

AgentS4D: Benchmarking Runtime Risks across the Execution Lifecycle of LLM-Based Workspace Agents

Evaluated models: GPT-5.5, Gemini 3.1 Pro, DeepSeek V4-Pro+2 more

GPT-5.5
Gemini 3.1 Pro
DeepSeek V4-Pro
MiniMax-M3
Qwen3.7-Plus

Incomplete MCP Tool-Call Attack Detection

Published Jul 28, 2026Analyzed Aug 13, 2026Source: arXiv

The MTGuard study evaluates unsafe Model Context Protocol tool calls originating from compromised server data, host-side execution changes, and malicious user-controlled resources. Its hybrid monitor combines pre-execution parameter inspection, behavioral observation, and post-execution result verification across browser-automation and financial-analysis agents.

Hybrid Analysis for Secure MCP Tool Use in LLM Agents

Evaluated models: GPT-5.6 Luna, DeepSeek V4 Flash, DeepSeek V4-Pro

Malicious GitHub Issue Injection Against Coding Agents

Published Jul 22, 2026Analyzed Aug 13, 2026Source: arXiv

IssueTrojanBench studies indirect prompt injection when a coding agent processes an apparently ordinary software-development issue or related artifact. Starting with six legitimate seed issues from two Python repositories, the authors construct 696 adversarial issue variants spanning four unsafe-action families and six delivery formats, then execute those variants across six agent-model configurations.

IssueTrojanBench: Benchmarking AI Coding Agents Against Malicious Issue Requests

Evaluated models: GPT-5.3 Codex, GPT-5.4, Claude Sonnet 4.6

Third-Party Agent Skill Supply-Chain Execution Risks

Published Jul 22, 2026Analyzed Aug 13, 2026Source: arXiv

OpenSkillRisk evaluates whether agent harnesses safely handle third-party skills that introduce risky behavior through otherwise plausible, benign tasks. The benchmark assembles 263 risky skills from public agent-skill ecosystems and tests three CLI-agent harnesses against seven risk categories using isolated task workspaces, mocked external services, and execution-level evidence.

OpenSkillRisk: Benchmarking Agent Safety When Using Real-World Risky Third-Party Skills

Evaluated models: GPT-5.1 Codex Mini, GPT-5.3 Codex, GPT-5.4+10 more

GPT-5.1 Codex Mini
GPT-5.3 Codex
GPT-5.4
Gemini 2.5 Pro
Gemini 3 Flash
Gemini 3.1 Pro
Claude Haiku 4.5
Claude Sonnet 4.6
Claude Opus 4.6
DeepSeek V3.2
DeepSeek V4-Pro
Kimi-K2.6
GLM-5.1

Prompt injection across agent privilege boundaries

Published Jul 21, 2026Analyzed Sep 9, 2026Source: arXiv

Untrusted issue descriptions and tool responses can redirect privileged coding and tool agents. Twin Agent separates exploration from execution and restricts the information exchanged between them.

Twin Agent: Context Residual Compression for Privilege Separated Agents

Evaluated models: GPT-5.2, Gemini 2.5 Flash, Gemini 3.5 Flash

Related research categories