Skip to main content

Ollama Judge (Local)

Run evaluations entirely locally with no API key required. Ideal for private data or offline environments.

Setup​

Install and start Ollama:

# Install Ollama
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama3.2
ollama pull mistral

# Ollama runs at http://localhost:11434 by default

Configuration​

AgentEvalConfig config = AgentEvalConfig.builder()
    .judgeModel(JudgeModels.ollama("llama3.2"))
    .build();

Custom endpoint:

JudgeModels.ollama("llama3.2", "http://my-ollama-server:11434")
ModelParametersQuality Notes
llama3.23B/8BGood general-purpose judge
mistral7BStrong instruction following
llama3.1:70b70BBest quality for local evals
qwen2.5:14b14BGood reasoning

YAML Configuration​

agenteval:
  judge:
    provider: ollama
    model: llama3.2
    endpoint: http://localhost:11434  # optional
tip

Ollama is the best choice when:

  • Your test data contains sensitive/private information
  • You want zero-cost evaluations (just compute)
  • You need deterministic, reproducible results