Skip to main content
This page’s examples describe released Agent Judge 0.17.0 and the fundamentals reactor. For configured Agent Eval 0.18 execution, use the requirements walkthrough and current API reference.
Every built-in judge returns the normalized Judgment model. Status is required, while a stored score and label are present only when that judge completed a measurement or classification. Reach for these in the order the tutorial does: the least interpretive instrument that can reliably answer the question. A criterion the compiler, JUnit, ArchUnit or JaCoCo already settles does not need a judge at all — it needs the judge only when its answer has to travel into a definition of done.

​
Core file-system judges

Artifact: agent-judge-core. These judges produce Boolean outcomes and therefore normally store no score. The executable starting point is tutorial module 01.
Changed in 0.16.0 — degenerate inputs no longer pass. An empty path, ., a directory, an absolute path outside the workspace, and a parent traversal are each refused rather than treated as a satisfied check. Previously some of these returned PASS without examining anything.This is the failure mode judges are most prone to: a broken judge fails open with a plausible result. A test that cannot run reports a failure, but a judge that examines nothing still returns a status, and PASS on an unexamined thing is indistinguishable from PASS on a verified one. If you have suites that were quietly passing on a mistyped or empty path, they will start failing — that is the fix working, not a regression.

​
Command and coverage judges

Artifact: agent-judge-exec. The two coverage judges answer different questions and are meant to be used together. CoverageImprovementJudge measures; CoveragePreservationJudge decides, against a tolerance you supply. Its normalized score floors at 0.0, so a one-point drop and a sixty-point drop score identically. The distance survives in the judgment’s improvementPp metadata and its coverage_improved check — read the parts, not the score. Both judges return ERROR when no JaCoCo report is present: the subject was never measured, and zero would be a real assessment rather than the absence of one. Tutorial module 03 runs all three cases. Build and command judges run real processes in the context workspace. Use the project wrapper when present and choose timeouts appropriate to untrusted work. Tutorial module 02 executes the Maven build path.

​
Semantic file judges

Artifact: agent-judge-file. These judges expect reference and actual file locations in the evaluation context as documented in the aggregate Javadoc.

​
LLM judge

Artifact: agent-judge-llm. For a framework-neutral composition, use ModelBackedJudge from agent-judge-ai-core with a JudgePromptTemplate, JudgeModel, and JudgmentClassifier. Tutorial module 07 runs that complete path with no credentials.

​
RAG judges

Artifact: agent-judge-rag. RagContext defines the rag.question, rag.context, and rag.answer metadata keys and the framework fallback conventions. When required evidence is absent, these judges abstain rather than inventing a score.

​
Framework evaluators

Evaluators are bridges rather than judges. They execute or adapt framework output into JudgmentContext, then apply an ordinary Judge or Jury. The credential-free bridge samples are Koog module 10 and LangChain4j module 11.

​
Requirements judges

New in 0.16.0. Artifact: agent-judge-ai-core.
EarsJudge and Rfc2119Judge read written acceptance criteria and architectural constraints back against the implementation. See Requirements judges for the roster and strict rollup, a BOM import, runnable examples, and the case study.