Skip to main content

​
Judges are executable acceptance criteria

A judge plays the role that an assertion plays in ordinary software. It receives the evidence from an agent run and produces an explicit outcome with reasoning. Agent Judge keeps that contract small: Judge is a functional interface with one judge(JudgmentContext) method. Lambdas work directly, metadata can be added through composition, and richer implementations can extend DeterministicJudge or LLMJudge.

​
Status is the required outcome

A Judgment records three independent facts. A Boolean decision stores its status without also storing 1.0 or 0.0. This avoids two fields representing the same fact and later disagreeing. effectiveScore() derives a numeric view when a voting strategy needs one. The constructor enforces the cross-field rules. ABSTAIN and ERROR cannot carry a score, ERROR cannot carry a label, and both require non-blank reasoning.

​
Errors are outcomes, not exception transport

ERROR means a judge did not complete its evaluation. The judgment carries human-readable reasoning but no Throwable. The original exception should be logged where it was caught. This keeps the result deterministic and portable while preserving framework-native diagnostics in the in-process JudgmentContext when a judge needs them.

​
Result metadata crosses boundaries

Result metadata accepts only values that preserve their meaning across JSON and process boundaries. Supported values are strings, booleans, interoperable integers, finite numbers, arrays, and string-keyed objects, recursively. The result constructor normalizes, copies, and freezes this graph. SDK response objects, exceptions, enums, arbitrary-precision values, and Java time objects are rejected rather than failing later during serialization. Timing demonstrates the pattern. The portable result stores elapsedMillis, and Judgment.elapsed() derives a Java Duration view.

​
Framework-neutral does not mean dependency-free

The core API has no dependency on an agent runtime such as Spring AI, LangChain4j, Koog, or AgentClient. It does use Jackson, SLF4J, and JSpecify to implement serialization, logging, and declared nullness. Framework-specific bridges sit in separate modules. Applications add only the bridge and judge families that they need.

​
Composition separates policy from evidence

Judges produce individual evidence. Voting strategies decide how that evidence becomes a jury verdict. Status-counting policies such as majority and consensus reason directly about outcomes. Numeric policies such as weighted average and median use each eligible judgment’s effectiveScore(). Abstentions and errors are not silently converted into zero measurements; the selected strategy and error policy decide how they participate.

​
Cascades make cost explicit

A CascadedJury evaluates tiers in order. Cheap deterministic or structural checks can reject invalid work before an LLM-backed tier incurs latency and token use. The final tier is explicit, and each earlier tier declares whether it can reject or accept without escalation.

​
Examples remain executable

The separate Agent Judge Tutorial owns canonical examples. Its ten Maven modules compile and run without credentials, so documentation can link to maintained source instead of carrying divergent sample programs.