Cloudflare AI Gateway

Cloudflare AI Gateway is a proxy service that routes requests to AI providers through Cloudflare's infrastructure. It provides:

Caching - Reduce costs by caching identical requests
Rate limiting - Control request rates to avoid quota issues
Analytics - Track usage and costs across providers
Logging - Monitor requests and responses
Fallback - Configure fallback providers for reliability

The cloudflare-gateway provider lets you route your promptfoo evaluations through Cloudflare AI Gateway to any supported AI provider.

Provider Format

cloudflare-gateway:{provider}:{model}

Examples:

cloudflare-gateway:openai:gpt-5.2
cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929
cloudflare-gateway:groq:llama-3.3-70b-versatile

Required Configuration

Set your Cloudflare account ID and gateway ID:

export CLOUDFLARE_ACCOUNT_ID=your_account_id_here
export CLOUDFLARE_GATEWAY_ID=your_gateway_id_here

Provider API Keys

You need API keys for the providers you're routing through:

# For OpenAI
export OPENAI_API_KEY=your_openai_key

# For Anthropic
export ANTHROPIC_API_KEY=your_anthropic_key

# For Groq
export GROQ_API_KEY=your_groq_key

Using BYOK (Bring Your Own Keys)

If you've configured BYOK in Cloudflare, you can omit provider API keys entirely. Cloudflare will use the keys stored in your gateway configuration.

providers:
  # No OPENAI_API_KEY needed - Cloudflare uses stored key
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'

note

BYOK works best with OpenAI-compatible providers. Anthropic requires an API key because the SDK mandates it.

Authenticated Gateways

If your gateway has Authenticated Gateway enabled, you must provide the cfAigToken:

export CF_AIG_TOKEN=your_gateway_token_here

Basic Usage

promptfooconfig.yaml
prompts:
  - 'Answer this question: {{question}}'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.7

tests:
  - vars:
      question: What is the capital of France?

Supported Providers

Cloudflare AI Gateway supports routing to these providers:

Provider	Gateway Name	API Key Environment Variable
OpenAI	`openai`	`OPENAI_API_KEY`
Anthropic	`anthropic`	`ANTHROPIC_API_KEY`
Groq	`groq`	`GROQ_API_KEY`
Perplexity	`perplexity-ai`	`PERPLEXITY_API_KEY`
Google AI Studio	`google-ai-studio`	`GOOGLE_API_KEY`
Mistral	`mistral`	`MISTRAL_API_KEY`
Cohere	`cohere`	`COHERE_API_KEY`
Azure OpenAI	`azure-openai`	`AZURE_OPENAI_API_KEY`
Workers AI	`workers-ai`	`CLOUDFLARE_API_KEY`
Hugging Face	`huggingface`	`HUGGINGFACE_API_KEY`
Replicate	`replicate`	`REPLICATE_API_KEY`
Grok (xAI)	`grok`	`XAI_API_KEY`

note

AWS Bedrock is not supported through Cloudflare AI Gateway because it requires AWS request signing, which is incompatible with the gateway proxy approach.

Configuration Options

Gateway Configuration

Option	Type	Description
`accountId`	string	Cloudflare account ID
`accountIdEnvar`	string	Custom environment variable for account ID (default: `CLOUDFLARE_ACCOUNT_ID`)
`gatewayId`	string	AI Gateway ID
`gatewayIdEnvar`	string	Custom environment variable for gateway ID (default: `CLOUDFLARE_GATEWAY_ID`)
`cfAigToken`	string	Optional gateway authentication token
`cfAigTokenEnvar`	string	Custom environment variable for gateway token (default: `CF_AIG_TOKEN`)

Azure OpenAI Configuration

Azure OpenAI requires additional configuration:

Option	Type	Description
`resourceName`	string	Azure OpenAI resource name (required)
`deploymentName`	string	Azure OpenAI deployment name (required)
`apiVersion`	string	Azure API version (default: `2024-12-01-preview`)

providers:
  - id: cloudflare-gateway:azure-openai:gpt-4
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      resourceName: my-azure-resource
      deploymentName: my-gpt4-deployment
      apiVersion: 2024-12-01-preview

Workers AI Configuration

Workers AI routes requests to Cloudflare's edge-deployed models. The model name is included in the URL path:

providers:
  - id: cloudflare-gateway:workers-ai:@cf/meta/llama-3.1-8b-instruct
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

Provider-Specific Options

All options from the underlying provider are supported. For example, when using cloudflare-gateway:openai:gpt-5.2, you can use any OpenAI provider options.

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      temperature: 0.8
      max_tokens: 1000
      top_p: 0.9

Examples

Multiple Providers

Compare responses from different providers, all routed through your Cloudflare gateway:

promptfooconfig.yaml
prompts:
  - 'Explain {{topic}} in simple terms.'

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:anthropic:claude-sonnet-4-5-20250929
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

  - id: cloudflare-gateway:groq:llama-3.3-70b-versatile
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'

tests:
  - vars:
      topic: quantum computing

Authenticated Gateway

If your AI Gateway requires authentication:

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountId: '{{env.CLOUDFLARE_ACCOUNT_ID}}'
      gatewayId: '{{env.CLOUDFLARE_GATEWAY_ID}}'
      cfAigToken: '{{env.CF_AIG_TOKEN}}'

Custom Environment Variables

Use custom environment variable names for different projects or environments:

providers:
  - id: cloudflare-gateway:openai:gpt-5.2
    config:
      accountIdEnvar: MY_CF_ACCOUNT
      gatewayIdEnvar: MY_CF_GATEWAY
      apiKeyEnvar: MY_OPENAI_KEY

Gateway URL Structure

The provider constructs the gateway URL in this format:

https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_id}/{provider}

For example, with accountId: abc123 and gatewayId: my-gateway, requests to OpenAI would be routed through:

https://gateway.ai.cloudflare.com/v1/abc123/my-gateway/openai

Benefits of Using AI Gateway

Cost Reduction Through Caching

AI Gateway can cache identical requests, reducing costs when you run the same prompts multiple times (common during development and testing).

Unified Analytics

View usage across all your AI providers in a single Cloudflare dashboard, making it easier to track costs and usage patterns.

Rate Limit Protection

AI Gateway can help manage rate limits by queuing requests, preventing your evaluations from failing due to provider rate limits.

Logging and Debugging

All requests and responses are logged in Cloudflare, making it easier to debug issues and audit AI usage.

Provider Format​

Required Configuration​

Provider API Keys​

Using BYOK (Bring Your Own Keys)​

Authenticated Gateways​

Basic Usage​

Supported Providers​

Configuration Options​

Gateway Configuration​

Azure OpenAI Configuration​

Workers AI Configuration​

Provider-Specific Options​

Examples​

Multiple Providers​

Authenticated Gateway​

Custom Environment Variables​

Gateway URL Structure​

Benefits of Using AI Gateway​

Cost Reduction Through Caching​

Unified Analytics​

Rate Limit Protection​

Logging and Debugging​

See Also​