Skip to main content


The scenarios configuration lets you group a set of data along with a set of tests that should be run on that data. This is useful for when you want to test a wide range of inputs with the same set of tests.


Let's take the example of a language translation app. We want to test whether the system can accurately translate three phrases ('Hello world', 'Good morning', and 'How are you?') from English to three different languages (Spanish, French, and German).

You're a translator.  Translate this into {{language}}: {{input}}
Speak in {{language}}: {{input}}

Instead of creating individual tests for each combination, we can create a scenarios that groups this data and the tests/assertions together:

- config:
- vars:
language: Spanish
expectedHelloWorld: 'Hola mundo'
expectedGoodMorning: 'Buenos días'
expectedHowAreYou: '¿Cómo estás?'
- vars:
language: French
expectedHelloWorld: 'Bonjour le monde'
expectedGoodMorning: 'Bonjour'
expectedHowAreYou: 'Comment ça va?'
- vars:
language: German
expectedHelloWorld: 'Hallo Welt'
expectedGoodMorning: 'Guten Morgen'
expectedHowAreYou: 'Wie geht es dir?'
- description: Translated Hello World
input: 'Hello world'
- type: similar
value: '{{expectedHelloWorld}}'
threshold: 0.90
- description: Translated Good Morning
input: 'Good morning'
- type: similar
value: '{{expectedGoodMorning}}'
threshold: 0.90
- description: Translated How are you?
input: 'How are you?'
- type: similar
value: '{{expectedHowAreYou}}'
threshold: 0.90

This will generate a matrix of tests for each language and input phrase combination, running the same set of assertions on each.

The full source behind this sample is in examples/multiple-translations-scenarios.


The scenarios configuration is an array of Scenario objects. Each Scenario has two main parts:

  • config: an array of vars objects. Each vars object represents a set of variables that will be passed to the tests.
  • tests: an array of TestCase objects. These are the tests that will be run for each set of variables in the config.

Here is the structure of a Scenario:

descriptionstringNoOptional description of what you're testing
configPartial<TestCase>[]YesAn array of variable sets. Each set will be run through the tests.
testsTestCase[]YesThe tests to be run on each set of variables.

This functionality allows you to easily run a wide range of tests without having to manually create each one. It also keeps your configuration file cleaner and easier to read.