Human evaluation is essential for maintaining quality control and oversight of your AI system’s outputs. Create structured workflows for human reviewers to rate and provide feedback on AI responses.

1

Navigate to Create Menu

Select Human from the create menu.

2

Define Reviewer Guidelines

Write clear guidelines for human reviewers. These instructions appear during the review process and should include:

  • What aspects to evaluate
  • How to assign ratings
  • Examples of good and bad responses

3

Choose Rating Format

Choose between two rating formats:

Binary (Yes/No) Simple binary evaluation

Scale Nuanced rating system for detailed quality assessment

4

Configure Pass Criteria

Configure two types of pass criteria:

Pass query Define criteria for individual evaluation metrics

Example: Pass if evaluation score > 0.8

Pass evaluator (%) Set threshold for overall evaluation across multiple entries

Example: Pass if 80% of entries meet the human evaluation criteria