Third-Party Agent Skill Supply-Chain Execution Risks
OpenSkillRisk evaluates whether agent harnesses safely handle third-party skills that introduce risky behavior through otherwise plausible, benign tasks. The benchmark assembles 263 risky skills from public agent-skill ecosystems and tests three CLI-agent harnesses against seven risk categories using isolated task workspaces, mocked external services, and execution-level evidence.
Evaluated models: GPT-5.1 Codex Mini, GPT-5.3 Codex, GPT-5.4+10 more
- GPT-5.1 Codex Mini
- GPT-5.3 Codex
- GPT-5.4
- Gemini 2.5 Pro
- Gemini 3 Flash
- Gemini 3.1 Pro
- Claude Haiku 4.5
- Claude Sonnet 4.6
- Claude Opus 4.6
- DeepSeek V3.2
- DeepSeek V4-Pro
- Kimi-K2.6
- GLM-5.1