Skip to main content

Envoy AI Gateway

Envoy AI Gateway is an open-source AI gateway that provides a unified proxy layer for accessing various AI model providers. It offers OpenAI-compatible endpoints.

Setup​

  1. Deploy and configure your Envoy AI Gateway following the official setup guide
  2. Configure your gateway URL either via environment variable or in your config
  3. Set up authentication if required by your gateway configuration

Provider Format​

The Envoy provider uses this format:

  • envoy:<model_name> - Connects to your gateway using the specified model name

Configuration​

Basic Configuration​

promptfooconfig.yaml
# yaml-language-server: $schema=https://promptfoo.dev/config-schema.json
providers:
  - id: envoy:my-model
    config:
      apiBaseUrl: 'https://your-envoy-gateway.com/v1'

With Environment Variable​

Set your gateway URL as an environment variable:

export ENVOY_API_BASE_URL="https://your-envoy-gateway.com"

Then use the provider without specifying the URL:

providers:
  - id: envoy:my-model

Authenticating via header​

Envoy authentication is usually done with an x-api-key header. Here's an example of how to configure that:

providers:
  - id: envoy:my-model
    config:
      apiBaseUrl: 'https://your-envoy-gateway.com/v1'
      headers:
        x-api-key: 'foobar'

See Also​