Code Scanning
Promptfoo Code Scanning uses AI agents to find LLM-related vulnerabilities in your codebase and helps you fix them before you merge. By focusing specifically on LLM-related vulnerabilities, it finds issues that more general security scanners might miss.
How It Works
The scanner examines code changes for common LLM security risks including prompt injection, PII exposure, and excessive agency. Rather than just analyzing the surface-level diff, it traces data flows deep into your codebase to understand how user inputs reach LLM prompts, how outputs are used, and what capabilities your LLM has access to.
This agentic approach catches subtle security issues that span multiple files, while maintaining a high signal-to-noise ratio to avoid alert fatigue.
Getting Started
GitHub Action
Automatically scan pull requests with findings posted as review comments. This is the recommended way to use the scanner if your code is on GitHub. Set up the GitHub Action →
CLI Command
Run scans locally or in any CI environment. Use the CLI →
Severity Levels
Findings are classified by severity to help you prioritize:
- Critical 🔴: Immediate security risk
- High 🟠: Significant issue
- Medium 🟡: Moderate risk
- Low 🔵: Minor issue
Configure minimum severity thresholds in your scan settings.
Custom Guidance
Tailor scans to your needs by providing custom guidance:
- Focus on specific vulnerability types or code areas
- Adjust severity levels based on your context
- Suggest fixes using your preferred libraries
- Skip known false positives
Example:
guidance: |
Ignore the /examples directory—it contains demo code only.
Treat potential PII exposure as critical.
For this app, sending proprietary code to OpenAI or Claude is not a vulnerability.
Use Zod schemas for validation when suggesting fixes.
Cloud and Enterprise
Scans run on Promptfoo Cloud by default. For organizations that need to run scans on their own infrastructure, code scanning is available in Promptfoo Enterprise On-Prem.