Vercel AI Gateway

Vercel AI Gateway provides a unified interface to access models through a single API. This provider uses the official Vercel AI SDK.

When tracing is enabled, Promptfoo automatically turns on the AI SDK's built-in tracing for text generation, streaming, structured output, and embeddings. SDK spans inherit the current evaluation trace, including direct provider calls that supply a traceparent. Prompt and response content are not recorded. If you call the SDK directly from a file:// custom provider, enable experimental_telemetry yourself; Promptfoo's trajectory assertions can normalize its tool-call spans from ai.toolCall.name plus the matching ai.toolCall.args, ai.toolCall.arguments, or ai.toolCall.input attributes.

Setup

1. Enable AI Gateway in your Vercel Dashboard
2. Get your API key from the AI Gateway settings
3. Set the VERCEL_AI_GATEWAY_API_KEY environment variable or specify apiKey in your config

export VERCEL_AI_GATEWAY_API_KEY=your_api_key_here

Usage

Provider Format

The Vercel provider uses the format: vercel:<provider>/<model>

providers:
  - vercel:openai/gpt-4o-mini
  - vercel:anthropic/claude-sonnet-5
  - vercel:google/gemini-2.5-flash

Embedding Models

For embedding models, use the embedding: prefix:

providers:
  - vercel:embedding:openai/text-embedding-3-small

Configuration

Basic Configuration

providers:
  - id: vercel:openai/gpt-4o-mini
    config:
      temperature: 0.7
      maxTokens: 1000

Full Configuration Options

providers:
  - id: vercel:openai/gpt-4o-mini
    config:
      # Authentication
      apiKey: '{{env.VERCEL_AI_GATEWAY_API_KEY}}'
      # Or omit apiKey and set apiKeyEnvar: CUSTOM_API_KEY_VAR

      # Model settings
      temperature: 0.7
      maxTokens: 2000
      topP: 0.9
      topK: 40
      frequencyPenalty: 0.5
      presencePenalty: 0.3
      stopSequences:
        - "\n\n"

      # Request settings
      timeout: 60000
      headers:
        Custom-Header: 'value'

      # Streaming
      streaming: true

When using Claude 5, omit temperature, topP, and topK.

Configuration Parameters

Parameter	Type	Description
apiKey	string	Vercel AI Gateway API key
apiKeyEnvar	string	Custom environment variable name for API key
temperature	number	Controls randomness; range depends on the model
maxTokens	number	Maximum number of tokens to generate
maxRetries	number	Retry attempts for a failed request
topP	number	Nucleus sampling parameter
topK	number	Top-k sampling parameter
frequencyPenalty	number	Penalizes frequent tokens
presencePenalty	number	Penalizes tokens based on presence
stopSequences	string[]	Sequences where generation stops
timeout	number	Request timeout in milliseconds
headers	object	Additional HTTP headers
streaming	boolean	Use the streaming API for text generation
responseSchema	object	JSON schema for structured output
baseUrl	string	Override the AI Gateway base URL

Structured Output

Generate structured JSON output by providing a JSON schema:

promptfooconfig.yaml

# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - id: vercel:openai/gpt-4o
    config:
      responseSchema:
        type: object
        properties:
          sentiment:
            type: string
            enum: [positive, negative, neutral]
          confidence:
            type: number
          keywords:
            type: array
            items:
              type: string
        required:
          - sentiment
          - confidence
          - keywords

prompts:
  - 'Analyze the sentiment of this text: {{text}}'

tests:
  - vars:
      text: 'I love this product!'
    assert:
      - type: javascript
        value: output.sentiment === 'positive'

Streaming

Use Vercel's streaming API for text generation. Promptfoo collects the chunks and runs assertions on the completed response:

providers:
  - id: vercel:anthropic/claude-sonnet-5
    config:
      streaming: true
      maxTokens: 2000

The provider normalizes the AI SDK's tool-calls and content-filter finish reasons to tool_calls and content_filter. Use the finish-reason assertion with these values, or stop and length, for both streaming and non-streaming responses.

Supported Providers

Query Vercel's public model catalog for IDs, capabilities, and pricing:

curl -fsS https://ai-gateway.vercel.sh/v1/models

The endpoint requires no authentication. See the Vercel AI Gateway documentation for response fields and filtering examples.

Embedding Models

Generate embeddings for text similarity, search, and RAG applications:

Set the embedding provider for the similar assertion under defaultTest.options.provider.embedding:

promptfooconfig.yaml

providers:
  - vercel:openai/gpt-5.6-luna

prompts:
  - 'Answer concisely: {{question}}'

defaultTest:
  options:
    provider:
      embedding:
        id: vercel:embedding:openai/text-embedding-3-small

tests:
  - vars:
      question: 'What is the capital of France?'
    assert:
      - type: similar
        value: Paris
        threshold: 0.8

Supported embedding models:

Provider	Example Models
OpenAI	openai/text-embedding-3-small, openai/text-embedding-3-large
Google	google/gemini-embedding-001, google/text-embedding-005
Cohere	cohere/embed-v4.0
Voyage	voyage/voyage-3.5, voyage/voyage-code-3

Examples

Multi-Provider Comparison

promptfooconfig.yaml

# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - id: vercel:openai/gpt-4o-mini
    config:
      temperature: 0.7
  - id: vercel:anthropic/claude-sonnet-5
    config:
      maxTokens: 1000
  - id: vercel:google/gemini-2.5-flash
    config:
      temperature: 0.7

prompts:
  - 'Explain {{concept}} in simple terms'

tests:
  - vars:
      concept: 'quantum computing'
    assert:
      - type: llm-rubric
        value: 'The response should be easy to understand'

JSON Response with Validation

promptfooconfig.yaml

# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - id: vercel:openai/gpt-4o
    config:
      responseSchema:
        type: object
        properties:
          summary:
            type: string
          topics:
            type: array
            items:
              type: string
          wordCount:
            type: integer
        required:
          - summary
          - topics
          - wordCount

prompts:
  - 'Analyze this article and return a structured summary: {{article}}'

tests:
  - vars:
      article: 'Long article text...'
    assert:
      - type: javascript
        value: 'Array.isArray(output.topics) && output.topics.length > 0'

Environment Variables

Variable	Description
VERCEL_AI_GATEWAY_API_KEY	API key for AI Gateway
AI_GATEWAY_API_KEY	Fallback from the shell environment when VERCEL_AI_GATEWAY_API_KEY is unset
VERCEL_AI_GATEWAY_BASE_URL	Override the AI Gateway URL

Troubleshooting

Common Issues

1. Authentication Failed: Ensure your VERCEL_AI_GATEWAY_API_KEY is set correctly
2. Model Not Found: Check that the provider/model combination is supported
3. Request Timeout: Increase the timeout configuration value

Debug Mode

Enable debug logging to see detailed request/response information:

LOG_LEVEL=debug promptfoo eval

Related Links

• Vercel AI SDK Documentation
• Vercel AI Gateway
• Supported Providers
• promptfoo Provider Guide