When scaling an LLM app, it's essential to be able to measure the impact of any prompt or model change. This guide shows how to use integrate promptfoo with CI/CD workflows to automatically evaluate test cases and ensure quality.
📄️ Github Actions
This guide describes how to automatically run a before vs. after evaluation of edited prompts using the promptfoo Github Action.
promptfoo can be integrated with test frameworks like Jest in order to evaluate prompts as part of existing testing and CI workflows.
promptfoo can be integrated with test frameworks like Mocha and assertion libraries like Chai in order to evaluate prompts as part of existing testing and CI workflows.
📄️ Python Notebook
For an example of using promptfoo in a Google Colab/Jupyter Notebook, see this notebook.