Language Model Security Database

MemSecBench follows malicious agent-memory content from initial write through persistence, retrieval, action selection, execution, and attempted selective repair. Its controlled Write–Execute–Forget protocol evaluates 310 human-reviewed cases across two harnesses, four memory backends, three model backends, and seven evidence-gated lifecycle checkpoints.

AgentS4D measures unsafe actions and state changes across complete workspace-agent executions rather than treating task completion or isolated model responses as safety evidence. Its 328 sandboxed cases introduce risky content through user requests, documents, web resources, tools, third-party skills, and persistent memory, then compare the same cases across four agent harnesses and five model backends.

OpenSkillRisk evaluates whether agent harnesses safely handle third-party skills that introduce risky behavior through otherwise plausible, benign tasks. The benchmark assembles 263 risky skills from public agent-skill ecosystems and tests three CLI-agent harnesses against seven risk categories using isolated task workspaces, mocked external services, and execution-level evidence.