Skip to main content
Released in Agent Eval 1.0.0; none of this is in 0.18.0. It needs agent-judge-ai-core 1.0.0.
Strict loading, whole-paragraph reading, the completeness rule and the linter are new in 1.0.0. The syntax itself is unchanged since 0.16.0. The EARS and RFC 2119 judges take their requirements from Markdown documents written before the code. This page is the format those documents must follow, what the readers do with it, and how to check a document while you write it. Every example below is checked by RequirementDocumentsGuideTest: the valid ones lint clean, and the refused one produces exactly the report shown under it. Documents are read as CommonMark. Fenced and indented code blocks, HTML blocks and thematic breaks are never read, so an example requirement inside a fence (like every example on this page) is documentation. Paragraphs inside lists and block quotes are read like any other, and their list and quote markers are not part of the text.

EARS acceptance criteria

A criterion is a level-3 heading ### UCn-ACm: Title, followed in the same entry by its requirement sentence. The entry ends at the next heading of any level or at the end of the document.
  • The sentence is the first paragraph of the entry that is not a metadata line, read in full. It must contain shall and end in ., ! or ? (closing brackets, quotes and inline markup may follow the punctuation).
  • Metadata lines begin with Covers:, Rule:, Rules:, Tests: or Status:, bold or plain. They are traceability, never the requirement, and they cannot stand in for a missing sentence.
  • Applies when: condition (also **Applies when:** condition) declares the optional applicability, once per criterion, anywhere in the entry. Declaring it is what makes “not applicable” an available answer for that criterion; an EARS trigger such as When, While or If inside the sentence never declares applicability.
  • Later paragraphs of the entry are prose and are not read.
Valid; it lints with no problem:

RFC 2119 constraints

A rule is a level-3 heading ### RULE-n or ### UCn-RULEm with nothing else on its line, followed in the same entry by exactly one keyword statement and exactly one reason.
  • The statement starts a line with a bold keyword: **MUST**, **MUST NOT**, **SHOULD**, **SHOULD NOT** or **MAY**. It continues through its paragraph up to the next keyword, **Reason:**, **Covers:** or Applies when: line, and must end in ., ! or ?.
  • **Reason:** gives the reason on its own line.
  • **Covers:** is optional traceability. Applies when: declares the optional applicability, as for criteria.
  • These lines may also sit in a list item or a block quote.
Valid; it lints with no problem:

Wrapped lines

A sentence or statement wrapped across lines is joined with single spaces, as Markdown renders a soft wrap. A hard break (two trailing spaces, or a trailing backslash) joins the same way, without its backslash. So UC1-AC1 above is read as one sentence ending in “send exactly one confirmation.”, and RULE-1 as “call the payment gateway from inside a database transaction.”

What a strict load refuses

Each problem is reported as file:line ID: reason, on the line where the entry, sentence or statement starts. Refused:
The linter reports:
fromStrict refuses this document. from would keep both UC2-AC1 sentences as written, and leave RULE-3 out for want of a reason.

Lint a specification while you write it

RequirementDocuments.lint(Path.of("spec")) returns a Report with the documents read, how many criteria and constraints loaded, and every problem. Files that hold no recognisable heading, such as a README, contribute nothing. One test keeps a specification repository valid; it fails with every problem listed:
From the command line, java -cp <agent-judge-ai-core and its dependencies> io.github.markpollack.judge.ai.requirements.RequirementDocuments spec prints the report and exits with status 1 when there is any problem. Source: REQUIREMENT_DOCUMENTS.md at v1.0.0; the examples are checked by RequirementDocumentsGuideTest in agent-judge-ai-core.