Skip to main content

​
Evaluate retrieval at scale

While the playground experience allows you to experiment and debug when retrieval is not working well, it is important to do this at scale across multiple inputs and with a set of defined metrics. Follow the steps given below to run a test and evaluate context retrieval.
1

Initiate prompt testing

Click on test for a prompt that has an attached context (as explained in the previous section).Test button
2

Select your test dataset

Select your dataset which has the required inputs.Dataset selection
3

Choose context evaluation source

For the context to evaluate, select the dynamic Context SourceDataset selection
4

Add retrieval quality evaluators

Select context specific evaluators - e.g. Context recall, context precision or context relevance and trigger the testContext evaluators
5

Review retrieved context results

Once the run is complete, the retrieved context column will be filled for all inputs.Variable linking
6

Examine detailed chunk information

View complete details of retrieved chunks by clicking on any entry.Retrieval details
7

Analyze evaluator feedback

Evaluator scores and reasoning for every entry can be checked under the evaluation tab. Use this to debug retrieval issues.Evaluator reasoning
By running experiments iteratively as you are making changes to your AI application, you can check for any regressions in the retrieval pipeline and continue to test for new test cases.