Skip to main content
The agent says it is done. Should you merge? Agent Eval (in the Agent Judge repository) is a Java 21 library for evaluating work against requirements set before execution. Configure a Judge with its requirement, runtime and evidence, then call judge(). Configure a Jury with the selected requirements and call vote(). Retain the complete result before deciding whether the application may rely on it. Current release: 0.18.0 (2026-10-08), on Maven Central under io.github.markpollack:agent-judge-*.
0.18 configures requirements and execution up front, returns complete results with a separate application reliance policy, runs native RFC 2119 and EARS requirements, and stores results as durable V6 records. It is a breaking release for 0.17 callers; see the migration guide. Getting started lists the modules most callers need. The separate fundamentals tutorial still uses released 0.17.0.

​
Requirements, evidence and execution

Keep using javac, JUnit and static analysis for questions they can settle. Use a model or agent when checking a written requirement needs interpretation. Prepared evidence is supplied explicitly; workspace, tools and timeout are configured on an investigative backend. EarsJudge and Rfc2119Judge assess one actual requirement. EarsJury and Rfc2119Jury assess a selected roster. A generated investigative roster enters its backend once for the whole roster; that backend can make several internal model and tool calls. A structured runtime executes once per requirement and retains those individual invocation owners. The PetClinic case study uses the same unchanged subject: The model-backed rows replay archived responses. Fresh replay verifies the configured public caller and retention, rather than establishing new model findings about the subject.

​
Keep the result and the reliance decision

A Judgment keeps PASS, FAIL, ABSTAIN, NOT_APPLICABLE and ERROR distinct. A Verdict additionally retains the roster, actual requirement associations, child attempts and original results. Native facts and invocation ownership stay attached; absent recorded usage remains unknown. A refused native result preserves its complete original, including its own Checks and invocations. Application policy is a separate decision. Withholding reliance on a PASS Verdict does not rewrite its conclusion. Producer AssertJ stages evaluate once, cache the complete result or failure, and can apply policy once. V6 storage retains the corresponding EvaluationResult. Retained reads and assertions execute no producers or policies.

Build the configured example

Add the 0.18.0 dependencies and provide requirements, evidence and runtime configuration.

Executable requirements

Run prepared and whole-roster callers without inference.

PetClinic case study

Preserve RFC13, EARS6 and EARS52 while separating assessment from merge policy.

Read and retain results

Understand conclusions, native evidence, refusals and V6 storage.

​
Resources

The source tree uses project-specific Business Source License 1.1 terms. Read its root LICENSE for the exact grant and restrictions; earlier releases retain their distributed license.