Skip to main content

​
0.17.0

​
Requirements judges: conditional criteria, and a roster that cannot be empty

The EARS and RFC 2119 judges, introduced in 0.16, read a written requirements document back against the implementation it describes. 0.17 changes both. EarsCriterion and Rfc2119Constraint gain an optional applicability clause. A criterion may declare when it does not apply; the prompt then offers NOT_APPLICABLE only for criteria that carry one, naming them. An excluded criterion leaves the roster’s denominator and is counted separately, with a mandatory reason. Excluding an unconditional criterion, or excluding one with no reason, is a protocol error rather than a finding, and is never counted as an authorized exclusion.
A requirements judge with no requirements no longer passes. EarsJudge.create and Rfc2119Judge.create reject an empty roster, and the rollup refuses one independently.A roster is a denominator, and a conjunctive rollup over an empty one is vacuously true: nothing failed, nothing was unestablished, so the implementation “satisfies the specification”. Released 0.16.0 returns PASS for that, and the result is indistinguishable in every stored field from a specification that was genuinely met.A stored PASS with a criteriaTotal or constraintsTotal of 0 identifies a run affected before this fix. Parsing is unchanged — a document that matches nothing still yields an empty list, because that is a fact about the document. The judge is what refuses.

​
An instrument failure is no longer a rejection

A jury whose own machinery breaks used to charge the subject for it. In one evaluation corpus, 20 of 39 runs had a jury that errored or abstained, and every one was recorded as a subject that did not pass. 0.17 separates the two. A strategy that throws or returns an aggregate it was not entitled to produce yields an ERROR aggregation_failed verdict marked undecided, with every judge’s own result intact. Machinery failure never supplies rejection evidence, not even under TREAT_AS_FAIL. NOT_APPLICABLE also becomes its own judgment status, distinct from ABSTAIN: a question that should not have been asked, excluded from the denominator and counted separately. Exclusion must be declared in advance, and every ERROR now carries a reasonCode naming its cause.

​
Reading a stored verdict later

Advanced, and only needed if you store verdicts and read them back. Verdict gains seats and decision — which judgment sat where and whether its key is a real identity, and whether this jury decided or adopted the outcome of a tier. Verdicts.interpret returns one Interpretation for any verdict, live or stored since 0.13, so consumers no longer each decide for themselves which status was a rejection. See Interpreting verdicts, and Migrating to 0.17 for the one intended compile break and the construction-time changes.

​
0.15.0

Agent Judge 0.15 is a compatibility-preserving dependency-alignment and release-reproducibility update. It keeps the 0.14 status-first Judgment API, the Java 21 baseline, and all ten published modules unchanged.

​
Consumer dependency alignment

  • Standalone consumers continue to resolve Jackson 2.21.6, with Jackson Annotations on its 2.21 release line.
  • agent-judge-llm and agent-judge-rag now declare Jackson 3.1.6 in their flattened POMs so consumers without the AgentWorks BOM receive the accepted Jackson 3 floor.
  • The provided Agent Client integration baseline advances to 0.26.0. Applications still choose and provide their Agent Client runtime.

​
Agent Sandbox Core

agent-judge-exec now uses agent-sandbox-core 0.10.0. That Core release protects environment values in ExecSpec.toString() and avoids forwarding the whole parent process environment when caller overrides are present. Agent Judge’s own execution paths construct specs with a command and timeout only; they do not set environment variables. This release does not add the Sandbox Docker or E2B modules.

​
Reproducible release evidence

  • Build and snapshot workflows are pinned to the reviewed Build Tools commit, and the Build workflow supports exact-SHA manual verification.
  • Every module publishes binary, source, and generated Javadoc JARs with the project-specific Business Source License terms.
  • Maven Central attaches one aggregate CycloneDX 1.6 SBOM to the parent coordinate. It is supporting inventory for the reactor and included dependency graph; provided-scope inventory is not a claim that every listed component enters an application’s runtime closure.
See the complete 0.15 release notes and the GitHub release.

​
0.14.0

Agent Judge 0.14 replaces the former sealed score hierarchy with a normalized, portable Judgment contract. This is a breaking pre-1.0 release, so consumers must recompile and follow the migration guide.

​
Result model

  • Every judgment has a required PASS, FAIL, ABSTAIN, or ERROR status.
  • A normalized score and a classification label are independent optional facts.
  • Boolean outcomes do not duplicate status as a stored 1.0 or 0.0; effectiveScore() supplies an intentional numeric view for aggregation.
  • Errors carry reasoned outcomes rather than live exceptions.

​
Portable evidence

  • Result metadata is validated as portable primitives, arrays, and string-keyed objects, then recursively copied and frozen.
  • Invalid values identify their exact metadata path.
  • Timing is stored as integer elapsedMillis, with Judgment.elapsed() providing the Java Duration view.
  • Model usage records separately reported input, output, reasoning, cache-creation, cache-read, and total token quantities without deriving price.

​
Aggregation

  • Every voting strategy exposes an ErrorPolicy, defaulting to PROPAGATE.
  • Aggregates contain structured population and error-accounting evidence.
  • Consensus returns ABSTAIN when applicable judges disagree, leaving rejection or escalation to a downstream gate or tier.

​
API and dependencies

  • The io.github.markpollack.judge.score package and ReactiveJudge are removed.
  • The core remains agent-runtime-neutral and declares its real Jackson Databind, SLF4J API, and JSpecify dependencies.
  • JSpecify exposes optional score and label nullness in the Java API, with NullAway enforcing the adopted packages.

​
Framework baselines

The release candidate is verified against the final stable framework matrix: Framework bridge dependencies remain provided where practical so applications select their runtime.

​
Samples and reference

  • The Agent Judge Tutorial now owns ten canonical credential-free Maven modules.
  • Executable Koog and LangChain4j modules replace the former README-only bridge samples.
  • Aggregate Javadoc is warning-clean, warnings are release-breaking, and every published module attaches a Javadoc JAR.
  • See the complete 0.14 release notes.

​
Release packaging

  • The current source tree uses the project-specific Business Source License terms in the repository’s root LICENSE. Earlier published releases keep the licenses shipped with those releases.
  • Maven Central publication attaches one CycloneDX 1.6 aggregate SBOM to the parent coordinate, covering the ten reactor modules and their included external dependencies.
  • The exact-pinned release workflow runs the full test-bearing verification before deploy and uses the curated RELEASE_NOTES_0.14.md body.
  • The GitHub Release is notes and source archives only; the signed Maven artifacts and aggregate SBOM belong on Maven Central, with no separately uploaded GitHub release assets.

​
0.13.0

  • Upgraded to Spring AI 2.0.0 GA and its supported Spring Boot 4.1.0 baseline.
  • Added an OWASP dependency-check gate.

​
0.12.0

  • Aligned Jackson dependencies for the release line.

​
0.11.0

  • Added agent-judge-ai-core and framework-neutral model-backed judge composition.

​
0.10.0

  • Added Spring AI, LangChain4j, Koog, and AgentClient bridges.
  • Added RAG evaluation judges and the nine-module architecture that preceded agent-judge-ai-core.

​
0.9.x

  • Moved public coordinates to io.github.markpollack.
  • Added superset project comparison and the initial public release.