Published 8/3/2026
Analyzed 8/13/2026
VulnGym measures whether coding agents can locate and explain repository-level security vulnerabilities from realistic advisory and source-code context. The benchmark contains 184 reviewed advisories, 408 line-annotated vulnerability entries, and 23 repositories, with separate end-to-end detection and oracle-conditioned localization tasks.
VulnGym: Benchmarking Coding Agents for Repository-Level Vulnerability Detection
Evaluated models: DeepSeek V4 Flash, GLM 5.2, MiniMax-M3 +4 more
Source: arXiv