Anthropic Judge

Use Anthropic Claude models as the evaluation judge.

Setup

export ANTHROPIC_API_KEY=sk-ant-...

Configuration

AgentEvalConfig config = AgentEvalConfig.builder()
    .judgeModel(JudgeModels.anthropic("claude-haiku-4-5-20251001"))
    .build();

Recommended Models

Model	Speed	Cost	Quality
claude-haiku-4-5-20251001	Fast	Low	Good for most evals
claude-sonnet-4-6	Medium	Medium	Excellent reasoning
claude-opus-4-6	Slow	High	Best quality

Example

AgentEvalConfig.builder()
    .judgeModel(JudgeModels.anthropic("claude-sonnet-4-6"))
    .cacheResults(true)
    .build();

YAML Configuration

agenteval:
  judge:
    provider: anthropic
    model: claude-sonnet-4-6