Skip to main content
What’s New in 0.15 → Agent Judge is a Java 21 evaluation library for deciding whether an agent result satisfies its goal. It turns a definition of done into executable evidence: builds and commands, coverage and class versions, semantic file checks, RAG evaluation, bounded LLM assessment, and independent judges combined through explicit jury policies. It asks whether the work meets its requirements—not whether it resembles one reference answer. The central result is status-first. Every Judgment has a PASS, FAIL, ABSTAIN, or ERROR status, while a normalized score and classification label are independent optional facts. Result metadata is restricted to portable values so judgments can cross JSON and process boundaries reliably.

​
What you can verify

Prove the code works

Run commands and builds, check bytecode versions, and preserve or improve test coverage.

Inspect the resulting workspace

Check files and compare Java, Maven, XML, and text by structure or meaning rather than bytes alone.

Evaluate semantic quality

Apply criteria-based model judges and RAG checks for correctness, faithfulness, relevance, and hallucination.

Combine independent evidence

Use majority, consensus, average, weighted-average, or median policies while retaining every judge’s reasoning.

Escalate only when needed

Put cheap deterministic checks before slower execution or model-backed tiers with CascadedJury.

Keep uncertainty visible

Distinguish ABSTAIN and evaluator ERROR from a real negative finding instead of collapsing everything into a score.

​
Bring the same evaluation policy to each runtime

Spring AI, LangChain4j, Koog, and AgentClient each have their own execution model. Each Agent Judge bridge converts framework output into a JudgmentContext, after which the same judges and juries evaluate it. Adapters preserve native output, model identity, token usage, timing, and tool evidence in a shared JudgmentContext. The judges and jury policy above that boundary do not change when the producing runtime does.

​
Install 0.15

Add only the family and bridge modules that your application uses.

​
Learn with executable examples

The Agent Judge Tutorial progresses from one deterministic acceptance check to composed judges, voting and cascaded juries, custom and model-backed judges, and framework bridges. Every example runs without credentials.

Getting Started

Add Agent Judge 0.15 and run the first evaluation

Executable Tutorial

Run the evaluation patterns as maintained Java projects

Result and Jury Design

Understand status-first outcomes and aggregation

Research Foundations

See why evidence, requirements, cascades, and independent checks shape the design

​
Resources

​
License

The current Agent Judge source tree uses project-specific Business Source License 1.1 terms. Read the repository’s root LICENSE for the exact grant, restrictions, change date, and change license; earlier published releases remain governed by the license distributed with those releases.