Azure
The azureopenai
provider is an interface to OpenAI through Azure. It behaves the same as the OpenAI provider.
Setup
First, set the AZURE_OPENAI_API_KEY
environment variable.
Next, edit the promptfoo configuration file to point to the Azure provider.
azureopenai:chat:<deployment name>
- uses the given deployment (for chat endpoints such as gpt-35-turbo, gpt-4)azureopenai:completion:<deployment name>
- uses the given deployment (for completion endpoints such as gpt-35-instruct)
Also set the apiHost
value to point to your endpoint:
providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
Environment Variables
The Azure OpenAI provider uses the following environment variables:
Variable | Description | Required |
---|---|---|
AZURE_OPENAI_API_KEY | Your Azure OpenAI API key | Yes |
AZURE_OPENAI_API_HOST | API host | No |
AZURE_OPENAI_API_BASE_URL | API base URL | No |
AZURE_OPENAI_BASE_URL | Alternative API base URL | No |
Note: You only need to set one of AZURE_OPENAI_API_HOST
, AZURE_OPENAI_API_BASE_URL
, or AZURE_OPENAI_BASE_URL
. If multiple are set, the provider will use them in that order of preference.
Configuration
The YAML configuration can override environment variables and set additional parameters:
providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
temperature: 0.5
max_tokens: 1024
All other OpenAI provider environment variables and configuration properties are supported.
Using client credentials
To use client credentials for authentication with Azure, first install the peer dependency:
npm i @azure/identity
Then set the following configuration variables:
providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
azureClientId: 'your-azure-client-id'
azureClientSecret: 'your-azure-client-secret'
azureTenantId: 'your-azure-tenant-id'
azureAuthorityHost: 'https://login.microsoftonline.com' # Optional
azureTokenScope: 'https://cognitiveservices.azure.com/.default' # Optional
These credentials will be used to obtain an access token for the Azure OpenAI API.
The azureAuthorityHost
defaults to 'https://login.microsoftonline.com' if not specified. The azureTokenScope
defaults to 'https://cognitiveservices.azure.com/.default', the scope required to authenticate with Azure Cognitive Services.
You must also install a peer dependency from Azure:
npm i @azure/identity
Model-graded tests
Model-graded assertions such as factuality
or llm-rubric
use OpenAI by default. If you are using Azure, you must override the grader to point to your Azure deployment.
The easiest way to do this for all your test cases is to add the defaultTest
property to your config:
defaultTest:
options:
provider:
id: azureopenai:chat:gpt-4-deployment-name
config:
apiHost: 'xxxxxxx.openai.azure.com'
However, you can also do this for individual assertions:
# ...
assert:
- type: llm-rubric
value: Do not mention that you are an AI or chat assistant
provider:
id: azureopenai:chat:xxxx
config:
apiHost: 'xxxxxxx.openai.azure.com'
Or individual tests:
# ...
tests:
- vars:
# ...
options:
provider:
id: azureopenai:chat:xxxx
config:
apiHost: 'xxxxxxx.openai.azure.com'
assert:
- type: llm-rubric
value: Do not mention that you are an AI or chat assistant
The maybeEmitAzureOpenAiWarning
function in the implementation will warn you if you're using model-graded assertions with Azure providers without specifying an override.
Similarity
The similar
assertion type requires an embedding model such as text-embedding-ada-002
. Be sure to specify a deployment with an embedding model, not a chat model, when overriding the grader.
AI Services
You may also specify deployment_id
and dataSources
, used to integrate with the Azure AI Search API.
providers:
- id: azureopenai:chat:deploymentNameHere
config:
apiHost: 'xxxxxxxx.openai.azure.com'
deployment_id: 'abc123'
dataSources:
- type: AzureCognitiveSearch
parameters:
endpoint: '...'
key: '...'
indexName: '...'
(The inconsistency in naming convention between deployment_id
and dataSources
reflects the actual naming in the Azure API.)
Configuration
These properties can be set under the provider config
key`:
General config
Name | Description |
---|---|
apiHost | API host. |
apiBaseUrl | Base URL of the API (used instead of host). |
apiKey | API key. |
apiVersion | API version. |
Azure-specific config
Name | Description |
---|---|
azureClientId | Azure identity client ID. |
azureClientSecret | Azure identity client secret. |
azureTenantId | Azure identity tenant ID. |
azureAuthorityHost | Azure identity authority host. |
azureTokenScope | Azure identity token scope. |
deployment_id | Azure cognitive services deployment ID. |
dataSources | Azure cognitive services parameter for specifying data sources. |
OpenAI config:
Name | Description |
---|---|
temperature | Controls randomness of the output. |
top_p | Controls nucleus sampling. |
frequency_penalty | Penalizes new tokens based on their frequency. |
presence_penalty | Penalizes new tokens based on their presence. |
best_of | Generates multiple outputs and chooses the best. |
functions | Specifies functions available for use. |
function_call | Controls automatic function calling. |
response_format | Specifies the format of the response. |
stop | Specifies stop sequences for the generation. |
passthrough | Anything under passthrough will be sent as a top-level request param |
Assistants
To eval an OpenAI assistant on Azure, first create a deployment for the assistant and create an assistant in the Azure web UI.
Then install the peer dependency locally:
npm i @azure/openai-assistants
Next, record the assistant ID and set up your provider like so:
providers:
- id: azureopenai:assistant:asst_E4GyOBYKlnAzMi19SZF2Sn8I
config:
apiHost: yourdeploymentname.openai.azure.com
Be sure to replace the assistant ID and the name of your deployment.
Here's an example of a simple full assistant eval:
prompts:
- 'Write a tweet about {{topic}}'
providers:
- id: azureopenai:assistant:asst_E4GyOBYKlnAzMi19SZF2Sn8I
config:
apiHost: yourdeploymentname.openai.azure.com
tests:
- vars:
topic: bananas
See the guide on How to eval OpenAI assistants for more information on how to compare different models, instructions, and more.