Prompt Optimization

​

When to use prompt optimization

• Improve prompt performance based on specific evaluation metrics
• Eliminate guesswork in prompt engineering
• Automatically generate better prompt versions from test data

​

Optimize a prompt

1

Navigate to a test run

Open your workspace and select a completed test run associated with a prompt.

2

Start prompt optimization

Click the Optimize prompt button on the run page to open the optimization dialog.

The button may be disabled if:

• The test run is still running or queued
• Required evaluators or models are not available

Hover over the button to see the specific reasons.

3

Configure optimization settings

Select your optimization preferences:

• Evaluators to prioritize: Choose which metrics to focus on during optimization
• Optimization iterations: Set how many improvement cycles to run

4

Run the optimization

Submit your configuration to start the optimization process. The system will:

• Analyze your current prompt and test results
• Generate improved prompt versions using AI models
• Test new versions against your dataset
• Iterate based on evaluator feedback

5

Review optimization results

Once complete, you’ll receive:

• Side-by-side comparison of original and optimized prompts
• Detailed reasoning for each change made
• Performance improvements across your chosen evaluators
• Suggestions for accepting or modifying the optimized version

6

Accept or iterate

Review the optimized prompt and choose to:

• Accept: Creates a new prompt version and links it to your runs
• Discard: Discard the optimized prompt and keep the original

7

Optimization results

After accepting an optimized prompt, you’ll have:

• New prompt version tailored to your success metrics
• Performance tracking showing improvements over time in run reports

You will also receive an email when your prompt optimization is complete.