# ACP Spring Boot Autoconfiguration Source: https://lab.pollack.ai/docs/acp-java-sdk/autoconfig Spring Boot autoconfiguration for the ACP Java SDK — auto-configured clients, agents, and transports with property-driven configuration. Spring Boot autoconfiguration for the [ACP Java SDK](/projects/acp-java-sdk). Provides auto-configured clients, agents, and transports with property-driven configuration. ## Quick Start Add the starter dependency: ```xml theme={null} org.springaicommunity acp-spring-boot-starter 0.11.1 ``` ### Client Configure the transport in `application.properties` and inject the client: ```properties theme={null} spring.acp.client.transport.stdio.command=java spring.acp.client.transport.stdio.args=-jar,my-agent.jar ``` ```java theme={null} @Component public class MyService { private final AcpSyncClient client; public MyService(AcpSyncClient client) { this.client = client; } public void run() { client.initialize(); var session = client.newSession(new NewSessionRequest(cwd, List.of())); var response = client.prompt(new PromptRequest(session.sessionId(), content)); } } ``` ### Agent Annotate a Spring bean with `@AcpAgent` and add handler methods: ```java theme={null} @Component @AcpAgent(name = "my-agent", version = "1.0") public class MyAgent { @Initialize public InitializeResponse initialize(InitializeRequest request) { return InitializeResponse.ok(); } @NewSession public NewSessionResponse newSession(NewSessionRequest request) { return new NewSessionResponse(UUID.randomUUID().toString(), null, null); } @Prompt public PromptResponse prompt(PromptRequest request, SyncPromptContext context) { context.sendMessage("Hello!"); return PromptResponse.endTurn(); } } ``` For stdio agents, redirect logging to stderr and keep the JVM alive: ```properties theme={null} spring.main.banner-mode=off spring.main.keep-alive=true ``` ## Configuration Properties ### Client | Property | Default | Description | | ------------------------------------------------------- | ----------- | ---------------------------------------------- | | `spring.acp.client.request-timeout` | `30s` | Request timeout | | `spring.acp.client.transport.type` | auto-detect | `stdio` or `websocket` | | `spring.acp.client.transport.stdio.command` | — | Command to launch agent process | | `spring.acp.client.transport.stdio.args` | — | Command arguments (comma-separated) | | `spring.acp.client.transport.stdio.env.*` | — | Environment variables for the process | | `spring.acp.client.transport.websocket.uri` | — | WebSocket URI (e.g. `ws://localhost:8080/acp`) | | `spring.acp.client.transport.websocket.connect-timeout` | `10s` | WebSocket connection timeout | | `spring.acp.client.capabilities.read-text-file` | `true` | Advertise file read capability | | `spring.acp.client.capabilities.write-text-file` | `true` | Advertise file write capability | | `spring.acp.client.capabilities.terminal` | `false` | Advertise terminal capability | ### Agent | Property | Default | Description | | ---------------------------------- | ------- | ------------------------------ | | `spring.acp.agent.enabled` | `true` | Enable agent autoconfiguration | | `spring.acp.agent.request-timeout` | `60s` | Request processing timeout | | `spring.acp.agent.transport.type` | `stdio` | Transport type | ## Transport Selection The client transport is selected automatically based on which properties are set: * Set `spring.acp.client.transport.stdio.command` → stdio transport * Set `spring.acp.client.transport.websocket.uri` → WebSocket transport * Set `spring.acp.client.transport.type` → explicit selection (takes precedence) The agent defaults to stdio transport. Set `spring.acp.agent.enabled=false` to disable. ## Overriding Beans All auto-configured beans back off when you provide your own. Define a custom `AcpClientTransport`, `AcpSyncClient`, `AcpAsyncClient`, or `AcpAgentTransport` bean and the autoconfiguration will use yours instead. ## Tutorial Build an ACP agent as a Spring Boot application Use the autoconfigured ACP client in Spring Boot ## Requirements * Java 21+ * Spring Boot 4.0+ * ACP Java SDK 0.14.0+ ## Resources * [GitHub Repository](https://github.com/spring-ai-community/acp-autoconfig) — Source code * [ACP Java SDK](/projects/acp-java-sdk) — The underlying SDK * [Maven Central](https://central.sonatype.com/artifact/org.springaicommunity/acp-spring-boot-starter) — Published artifacts # ACP Java SDK Source: https://lab.pollack.ai/docs/acp-java-sdk/index A Java SDK for the Agent Client Protocol — build both clients and agents that work with Zed, JetBrains, VS Code, and any ACP-compliant editor. A pure Java implementation of the [Agent Client Protocol (ACP)](https://agentclientprotocol.com/) specification. Build clients that connect to ACP agents, or build agents that run inside code editors. ## How ACP Works ACP uses a subprocess model. A **client** (your application, or an editor like Zed) launches an **agent** as a child process and communicates over stdin/stdout using JSON-RPC messages. The protocol has three phases: 1. **Initialize** — client and agent exchange protocol versions and capabilities 2. **Session** — client creates a session with a working directory context 3. **Prompt** — client sends messages, agent streams back responses This is the same mechanism that Zed, JetBrains, and VS Code use to talk to AI agents. The SDK lets you build either side of that conversation. ## Overview The ACP Java SDK provides: * **Client SDK** — connect to and interact with any ACP-compliant agent * **Agent SDK** — build ACP-compliant agents that work in Zed, JetBrains, and VS Code * **Test utilities** — in-memory transports for fast, deterministic testing ## Quick Start ### Try it now The fastest way to see ACP in action is to clone the tutorial and run a module. The client example talks to [Gemini CLI](https://github.com/google-gemini/gemini-cli) (requires `GEMINI_API_KEY`). The agent example runs locally with no API key. ```bash theme={null} git clone https://github.com/markpollack/acp-java-tutorial.git cd acp-java-tutorial # Client: talk to Gemini as an ACP agent export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-01-first-contact # Agent: build and run your own (no API key needed) ./mvnw package -pl module-12-echo-agent -q ./mvnw exec:java -pl module-12-echo-agent ``` ### Client — Connect to an Agent This example launches [Gemini CLI](https://github.com/google-gemini/gemini-cli) as an ACP agent subprocess and sends it a prompt. Any CLI tool that speaks ACP over stdin/stdout works here — Gemini CLI is one such tool. `AgentParameters` builds the command line (`gemini --experimental-acp`). `StdioAcpClientTransport` spawns the process and handles the JSON-RPC framing. The `sessionUpdateConsumer` receives the agent's response text as it streams in — without it, you'd get the stop reason but no visible output. ```java theme={null} import com.agentclientprotocol.sdk.client.*; import com.agentclientprotocol.sdk.client.transport.*; import com.agentclientprotocol.sdk.spec.AcpSchema.*; import java.util.List; // Launch Gemini CLI as an ACP agent subprocess var params = AgentParameters.builder("gemini").arg("--experimental-acp").build(); var transport = new StdioAcpClientTransport(params); AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { // Print the agent's response as it streams in if (notification.update() instanceof AgentMessageChunk msg) { System.out.print(((TextContent) msg.content()).text()); } }) .build(); client.initialize(); var session = client.newSession(new NewSessionRequest(".", List.of())); var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("What is 2+2? Reply with just the number.")) )); System.out.println("\nStop reason: " + response.stopReason()); client.close(); // Output: 4 // Stop reason: END_TURN ``` Run this yourself: [Module 01: First Contact](https://github.com/markpollack/acp-java-tutorial/tree/main/module-01-first-contact) — full source with error handling and setup. ### Agent — Build Your Own This is the other side of the conversation. When you build an agent, **editors and clients launch your code** as a subprocess and send it JSON-RPC messages over stdin. Your agent handles three request types: initialize, new session, and prompt. The stdio transport reads from stdin and writes to stdout. `run()` blocks until the client disconnects. The module includes a demo client that launches this agent as a subprocess and exercises it — you'll see `Echo: Hello!` printed when you run it. ```java theme={null} import com.agentclientprotocol.sdk.annotation.*; import com.agentclientprotocol.sdk.agent.SyncPromptContext; import com.agentclientprotocol.sdk.agent.support.AcpAgentSupport; import com.agentclientprotocol.sdk.spec.AcpSchema.*; @AcpAgent class EchoAgent { @Initialize InitializeResponse init() { return InitializeResponse.ok(); } @NewSession NewSessionResponse newSession() { return new NewSessionResponse(UUID.randomUUID().toString(), null, null); } @Prompt PromptResponse prompt(PromptRequest req, SyncPromptContext ctx) { ctx.sendMessage("Echo: " + req.text()); return PromptResponse.endTurn(); } } // Reads JSON-RPC from stdin, writes to stdout — editors connect here AcpAgentSupport.create(new EchoAgent()) .transport(new StdioAcpAgentTransport()) .run(); ``` Run this yourself: [Module 12: Echo Agent](https://github.com/markpollack/acp-java-tutorial/tree/main/module-12-echo-agent) — includes a demo client that launches the agent and sends test prompts. No API key required. ### Adding the SDK to your project ```xml theme={null} com.agentclientprotocol acp-core 0.15.0 ``` For annotation-based agents (as shown above), add `acp-agent-support` instead — it includes `acp-core` transitively: ```xml theme={null} com.agentclientprotocol acp-agent-support 0.15.0 ``` Version `0.12.0` added session/list, session/close, session/resume, elicitation, session/fork, and session/set\_config\_option. Version `0.14.0` (superseding the never-released 0.13.0) adds logout, session/delete, additional workspace directories, per-chunk message IDs, and the unstable providers/\* configuration methods, promotes session/set\_config\_option to stable, and deprecates the removed session/set\_model API (use session/set\_config\_option with a `"model"` category option). See the [API Reference](/docs/acp-java-sdk/reference/java#installation) for installation. ## Three Agent API Styles | Style | Entry Point | Best For | | -------------------- | ---------------------- | --------------------------------------------- | | **Annotation-based** | `@AcpAgent`, `@Prompt` | Least boilerplate, declarative style | | **Sync** | `AcpAgent.sync()` | Blocking handlers, plain return values | | **Async** | `AcpAgent.async()` | Reactive applications, Project Reactor `Mono` | All three styles produce identical protocol behavior. Choose based on programming preference. ## Documentation Client API, Agent API (all three styles), protocol types, transports, errors 30-module progressive tutorial from client basics to IDE integration ## Resources * [GitHub Repository](https://github.com/agentclientprotocol/java-sdk) — Source code * [ACP Java Tutorial](https://github.com/markpollack/acp-java-tutorial) — 30 hands-on modules ### ACP Ecosystem * [Agent Client Protocol](https://agentclientprotocol.com/) — Official specification * [ACP Specification](https://agentclientprotocol.com/protocol/overview) — Protocol details (initialization, sessions, prompt turns) * [Agents Directory](https://agentclientprotocol.com/overview/agents) — ACP-compliant agents * [Clients Directory](https://agentclientprotocol.com/overview/clients) — Editors and clients that support ACP ### Other ACP SDKs * [Kotlin SDK](https://github.com/agentclientprotocol/kotlin-sdk) * [Python SDK](https://github.com/agentclientprotocol/python-sdk) * [TypeScript SDK](https://github.com/agentclientprotocol/typescript-sdk) * [Rust SDK](https://github.com/agentclientprotocol/rust-sdk) ### Editor Documentation * [Zed — External Agents](https://zed.dev/docs/ai/external-agents) * [JetBrains — ACP Support](https://www.jetbrains.com/help/ai-assistant/acp.html) * [VS Code — ACP Extension](https://github.com/formulahendry/vscode-acp) # Java API Reference Source: https://lab.pollack.ai/docs/acp-java-sdk/reference/java Complete API reference for the ACP Java SDK — client, agent, protocol types, transports, and test utilities. Complete API reference for the ACP Java SDK, covering client, agent (all three styles), protocol types, transports, errors, and test utilities. *** ## Installation ### Maven (0.15.0 — stable) Core SDK (client + sync/async agent APIs): ```xml theme={null} com.agentclientprotocol acp-core 0.15.0 ``` Annotation-based agent support (includes `acp-core` transitively): ```xml theme={null} com.agentclientprotocol acp-agent-support 0.15.0 ``` Test utilities: ```xml theme={null} com.agentclientprotocol acp-test 0.15.0 test ``` WebSocket server transport for agents: ```xml theme={null} com.agentclientprotocol acp-websocket-jetty 0.15.0 ``` ### Gradle ```groovy theme={null} // build.gradle implementation 'com.agentclientprotocol:acp-core:0.15.0' // Optional modules implementation 'com.agentclientprotocol:acp-agent-support:0.15.0' implementation 'com.agentclientprotocol:acp-websocket-jetty:0.15.0' testImplementation 'com.agentclientprotocol:acp-test:0.15.0' ``` ```kotlin theme={null} // build.gradle.kts implementation("com.agentclientprotocol:acp-core:0.15.0") // Optional modules implementation("com.agentclientprotocol:acp-agent-support:0.15.0") implementation("com.agentclientprotocol:acp-websocket-jetty:0.15.0") testImplementation("com.agentclientprotocol:acp-test:0.15.0") ``` ### Snapshot (0.15.0-SNAPSHOT) For unreleased features, add the snapshot repository and use the snapshot version: ```xml theme={null} central-snapshots https://central.sonatype.com/repository/maven-snapshots/ true false ``` Then use `0.16.0-SNAPSHOT` in place of `0.15.0` in your dependencies. *** ## Three Agent API Styles ### Quick Comparison | Feature | Annotation-based | Sync | Async | | :---------------- | :------------------------------------------- | :----------------------- | :-------------------------------- | | **Entry Point** | `@AcpAgent` class | `AcpAgent.sync()` | `AcpAgent.async()` | | **Handler Style** | Annotated methods | Lambda callbacks | Lambda callbacks returning `Mono` | | **Return Values** | Auto-converted (`String` → `PromptResponse`) | Direct protocol types | `Mono` | | **Boilerplate** | Lowest | Moderate | Moderate | | **Best For** | Most applications | Simple blocking handlers | Reactive applications | | **Runtime** | `AcpAgentSupport` | `AcpSyncAgent` | `AcpAsyncAgent` | All three produce identical protocol behavior and support the same capabilities. ### When to Use Each * **Annotation-based** — default choice. Least boilerplate, auto-converts return types, supports interceptors and custom argument resolvers. * **Sync** — when you want explicit control over every handler without annotations. Blocking void methods for sending updates. * **Async** — when your agent needs non-blocking I/O. Uses Project Reactor `Mono` for composable async chains. *** ## Client API ### `AcpClient` — Factory | Method | Return Type | Description | | :----------------- | :----------------------- | :----------------------------- | | `sync(transport)` | `AcpSyncClient.Builder` | Create blocking client builder | | `async(transport)` | `AcpAsyncClient.Builder` | Create reactive client builder | ### `AcpSyncClient` — Blocking Client | Method | Return Type | Description | | :-------------------------------- | :------------------------------- | :---------------------------------------------------------------------- | | `initialize()` | `InitializeResponse` | Protocol handshake with defaults | | `initialize(request)` | `InitializeResponse` | Handshake with custom capabilities | | `authenticate(request)` | `AuthenticateResponse` | Authenticate with a method ID | | `logout(request)` | `LogoutResponse` | Clear stored credentials *(0.14.0)* | | `newSession(request)` | `NewSessionResponse` | Create a new session | | `loadSession(request)` | `LoadSessionResponse` | Resume an existing session (replays history) | | `listSessions(request)` | `ListSessionsResponse` | List sessions, optional cwd filter *(0.12.0)* | | `resumeSession(request)` | `ResumeSessionResponse` | Reconnect to session without history replay *(0.12.0)* | | `closeSession(request)` | `CloseSessionResponse` | Close session and free resources *(0.12.0)* | | `deleteSession(request)` | `DeleteSessionResponse` | Permanently delete a stored session *(0.14.0)* | | `forkSession(request)` | `ForkSessionResponse` | Fork a session into a new branch *(0.12.0, unstable)* | | `setSessionConfigOption(request)` | `SetSessionConfigOptionResponse` | Set a session config value *(0.12.0)* | | `listProviders(request)` | `ListProvidersResponse` | List configurable model/backend providers *(0.14.0, unstable)* | | `setProvider(request)` | `SetProviderResponse` | Configure a provider (protocol, base URL, headers) *(0.14.0, unstable)* | | `disableProvider(request)` | `DisableProviderResponse` | Disable a provider by id *(0.14.0, unstable)* | | `prompt(request)` | `PromptResponse` | Send prompt, block until response | | `cancel(notification)` | `void` | Cancel current prompt (fire-and-forget) | | `getAgentCapabilities()` | `NegotiatedCapabilities` | Capabilities reported by agent | | `close()` | `void` | Close connection | ### Builder Configuration ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { // Handle streaming updates during prompt() }) .readTextFileHandler(req -> { // Agent requests a file read return new ReadTextFileResponse(Files.readString(Path.of(req.path()))); }) .writeTextFileHandler(req -> { // Agent requests a file write Files.writeString(Path.of(req.path()), req.content()); return new WriteTextFileResponse(); }) .requestPermissionHandler(req -> { // Agent requests permission return new RequestPermissionResponse(req.options().getFirst().id()); }) .createElicitationHandler(req -> { // Agent requests structured user input (unstable) return CreateElicitationResponse.accept(Map.of("choice", "option-a")); }) .build(); ``` ### Example — Complete client lifecycle This launches Gemini CLI as an ACP agent subprocess and sends it a prompt. `AgentParameters` builds the command line; `StdioAcpClientTransport` spawns the process and handles JSON-RPC framing over stdin/stdout. ```java theme={null} // Launch "gemini --experimental-acp" as a subprocess var params = AgentParameters.builder("gemini") .arg("--experimental-acp") .build(); var transport = new StdioAcpClientTransport(params); AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { var update = notification.update(); if (update instanceof AgentMessageChunk msg) { System.out.print(((TextContent) msg.content()).text()); } }) .build(); client.initialize(); var session = client.newSession(new NewSessionRequest("/workspace", List.of())); var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Hello, world!")) )); System.out.println("Stop reason: " + response.stopReason()); client.close(); ``` *** ## Agent API — Annotation-Based The `acp-agent-support` module provides a declarative programming model using annotations. ### Annotations #### Class-Level | Annotation | Description | | ----------- | ------------------------------------------------------------------------ | | `@AcpAgent` | Marks a class as an ACP agent. Optional `name` and `version` attributes. | #### Handler Methods | Annotation | JSON-RPC Method | Description | | ------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------ | | `@Initialize` | `initialize` | Protocol initialization and capability negotiation | | `@Logout` | `logout` | Clears stored credentials *(0.14.0)* | | `@NewSession` | `session/new` | Creates a new agent session | | `@LoadSession` | `session/load` | Loads an existing session by ID (replays history) | | `@ListSessions` | `session/list` | Lists sessions with optional cwd filter *(0.12.0)* | | `@ResumeSession` | `session/resume` | Reconnects to session without history replay *(0.12.0)* | | `@CloseSession` | `session/close` | Closes session and frees resources *(0.12.0)* | | `@DeleteSession` | `session/delete` | Permanently deletes a stored session *(0.14.0)* | | `@ForkSession` | `session/fork` | Forks a session into a new branch *(0.12.0, unstable)* | | `@SetSessionConfigOption` | `session/set_config_option` | Sets a session config value *(0.12.0)* | | `@ListProviders` | `providers/list` | Lists configurable providers *(0.14.0, unstable)* | | `@SetProvider` | `providers/set` | Configures a provider *(0.14.0, unstable)* | | `@DisableProvider` | `providers/disable` | Disables a provider by id *(0.14.0, unstable)* | | `@Prompt` | `session/prompt` | Handles user prompts | | `@SetSessionMode` | `session/set_mode` | Changes operational mode | | `@SetSessionModel` | `session/set_model` | **Deprecated** — removed from the spec; use `@SetSessionConfigOption` with a `"model"` category option | | `@Cancel` | `session/cancel` | Cancellation notification (fire-and-forget) | > **Deprecated: the session-model API (0.14.0).** `session/set_model` and the related types > (`@SetSessionModel`, `SetSessionModelRequest`/`Response`, `SessionModelState`, `ModelInfo`, and the > `models` field on session responses) were removed from the ACP spec in June 2026 and are marked > `@Deprecated(forRemoval = true)`. They still work for now but will be removed in a future release. > Expose model selection through `session/set_config_option` instead: advertise a `select` config > option whose `category` is `"model"`, and switch models with `setSessionConfigOption(...)`. This is > the same mechanism used for session modes (`category: "mode"`) and reasoning level > (`category: "thought_level"`). #### Parameter Annotations | Annotation | Description | | --------------- | ------------------------------------------ | | `@SessionId` | Injects the current session ID as `String` | | `@SessionState` | Injects session-specific state | ### Flexible Method Signatures Handler methods support flexible parameter resolution: ```java theme={null} // Minimal @Initialize InitializeResponse init() { return InitializeResponse.ok(); } // With request @Prompt PromptResponse answer(PromptRequest req) { ... } // With context @Prompt PromptResponse answer(PromptRequest req, SyncPromptContext ctx) { ... } // Auto-converted return types @Prompt String simpleAnswer(PromptRequest req) { ... } // → PromptResponse.text(value) @Prompt void streaming(PromptRequest req, SyncPromptContext ctx) { ... } // → endTurn() ``` ### Return Value Handling | Return Type | Conversion | | ---------------------- | ----------------------------------------- | | Protocol response type | Passed through directly | | `String` | Converted to `PromptResponse.text(value)` | | `void` | Converted to `PromptResponse.endTurn()` | | `Mono` | Unwrapped and returned | ### `SyncPromptContext` Available in `@Prompt` handlers. Provides blocking methods for agent-client interaction: ```java theme={null} @Prompt PromptResponse handle(PromptRequest req, SyncPromptContext ctx) { // Session info String sessionId = ctx.getSessionId(); NegotiatedCapabilities caps = ctx.getClientCapabilities(); // Messages and thoughts ctx.sendMessage("Working on it..."); ctx.sendThought("Let me analyze this..."); // Tag streamed chunks with a message ID (0.14.0) — chunks sharing an id // form one logical message; a new id starts a new message ctx.sendMessage("First part...", "msg-1"); ctx.sendThought("Reasoning...", "msg-1"); // File operations (requires client capabilities) String content = ctx.readFile("/path/to/file.txt"); ctx.writeFile("/path/to/output.txt", "content"); Optional maybe = ctx.tryReadFile("/path/to/file.txt"); // Permissions boolean allowed = ctx.askPermission("Delete files in /tmp?"); String choice = ctx.askChoice("Which format?", "JSON", "XML", "YAML"); // Terminal execution (requires client capabilities) CommandResult result = ctx.execute("ls", "-la"); return PromptResponse.endTurn(); } ``` ### `AcpAgentSupport` — Bootstrap ```java theme={null} AcpAgentSupport.create(new MyAgent()) .transport(StdioAcpAgentTransport.create()) .requestTimeout(Duration.ofSeconds(60)) // Optional .interceptor(new LoggingInterceptor()) // Optional .argumentResolver(new UserResolver()) // Optional .returnValueHandler(new FutureHandler()) // Optional .run(); // Blocks until client disconnects ``` ### Interceptors Cross-cutting concerns like logging, metrics, or error handling: ```java theme={null} public class LoggingInterceptor implements AcpInterceptor { @Override public boolean preInvoke(AcpInvocationContext context) { log.info("Invoking: {}", context.getAcpMethod()); return true; // Continue processing } @Override public Object postInvoke(AcpInvocationContext context, Object result) { log.info("Result: {}", result); return result; } @Override public int getOrder() { return 0; } // Lower values execute first } ``` ### Example — Complete annotation-based agent ```java theme={null} @AcpAgent(name = "code-assistant", version = "1.0.0") class CodeAssistant { private final Map> sessionHistory = new ConcurrentHashMap<>(); @Initialize InitializeResponse init() { return InitializeResponse.ok(); } @NewSession NewSessionResponse newSession(NewSessionRequest req) { String sessionId = UUID.randomUUID().toString(); sessionHistory.put(sessionId, new ArrayList<>()); return new NewSessionResponse(sessionId, List.of(), List.of()); } @Prompt PromptResponse prompt(PromptRequest req, SyncPromptContext ctx) { ctx.sendThought("Analyzing the code..."); if (ctx.getClientCapabilities().supportsReadTextFile()) { ctx.sendMessage("I can access files if needed."); } ctx.sendMessage("Here's my analysis..."); return PromptResponse.endTurn(); } @Cancel void onCancel(CancelNotification notification, @SessionId String sessionId) { sessionHistory.remove(sessionId); } } ``` *** ## Agent API — Sync (Builder) Blocking handlers with plain return values. No annotations. ### Builder Methods | Method | Description | | :--------------------------------------- | :------------------------------------------------------- | | `initializeHandler(handler)` | Handle `initialize` requests | | `authenticateHandler(handler)` | Handle `authenticate` requests | | `logoutHandler(handler)` | Handle `logout` requests *(0.14.0)* | | `newSessionHandler(handler)` | Handle `session/new` requests | | `loadSessionHandler(handler)` | Handle `session/load` requests | | `listSessionsHandler(handler)` | Handle `session/list` requests *(0.12.0)* | | `resumeSessionHandler(handler)` | Handle `session/resume` requests *(0.12.0)* | | `closeSessionHandler(handler)` | Handle `session/close` requests *(0.12.0)* | | `deleteSessionHandler(handler)` | Handle `session/delete` requests *(0.14.0)* | | `forkSessionHandler(handler)` | Handle `session/fork` requests *(0.12.0, unstable)* | | `setSessionConfigOptionHandler(handler)` | Handle `session/set_config_option` requests *(0.12.0)* | | `listProvidersHandler(handler)` | Handle `providers/list` requests *(0.14.0, unstable)* | | `setProviderHandler(handler)` | Handle `providers/set` requests *(0.14.0, unstable)* | | `disableProviderHandler(handler)` | Handle `providers/disable` requests *(0.14.0, unstable)* | | `promptHandler(handler)` | Handle `session/prompt` requests | | `cancelHandler(handler)` | Handle `session/cancel` notifications | ### Example ```java theme={null} AcpSyncAgent agent = AcpAgent.sync(transport) .initializeHandler(req -> InitializeResponse.ok()) .newSessionHandler(req -> new NewSessionResponse(UUID.randomUUID().toString(), null, null)) .promptHandler((req, context) -> { context.sendMessage("Hello!"); return PromptResponse.endTurn(); }) .build(); agent.run(); // Blocks until client disconnects ``` ### Prompt Handler Context The `context` parameter in `promptHandler` provides: | Method | Description | | :------------------------------ | :------------------------ | | `getSessionId()` | Current session ID | | `sendMessage(text)` | Send `AgentMessageChunk` | | `sendThought(text)` | Send `AgentThoughtChunk` | | `sendUpdate(sessionId, update)` | Send any `SessionUpdate` | | `readFile(path, offset, limit)` | Read file from client | | `writeFile(path, content)` | Write file on client | | `requestPermission(request)` | Ask client for permission | | `getClientCapabilities()` | Check client capabilities | *** ## Agent API — Async (Builder) Reactive handlers returning `Mono`. Uses Project Reactor. ### Example ```java theme={null} AcpAsyncAgent agent = AcpAgent.async(transport) .initializeHandler(req -> Mono.just(InitializeResponse.ok())) .newSessionHandler(req -> Mono.just(new NewSessionResponse( UUID.randomUUID().toString(), null, null))) .promptHandler((req, context) -> context.sendMessage("Hello!") .then(Mono.just(PromptResponse.endTurn()))) .build(); agent.start().then(agent.awaitTermination()).block(); ``` The async context's `sendMessage()`, `sendUpdate()`, etc. return `Mono`, composable with `.then()` and `.flatMap()`. *** ## Convenience Methods vs Full API The SDK provides convenience methods that cover the most common operations. Use these by default — they produce cleaner code and handle the protocol details for you. ### When convenience methods are enough (\~80% of cases) ```java theme={null} // Response factories InitializeResponse.ok() // default capabilities InitializeResponse.ok(customCapabilities) // custom capabilities PromptResponse.endTurn() // stop reason END_TURN PromptResponse.text("response") // message + endTurn in one call // Sending updates (on SyncPromptContext or async equivalent) context.sendMessage("Hello"); // AgentMessageChunk with TextContent context.sendThought("Analyzing..."); // AgentThoughtChunk with TextContent // File operations String content = context.readFile("pom.xml"); // read with defaults context.writeFile("output.txt", "content"); // write file // Terminal CommandResult result = context.execute("ls", "-la"); // Permissions boolean ok = context.askPermission("Delete temp files?"); String choice = context.askChoice("Format?", "JSON", "XML", "YAML"); ``` ### When to use the full API (\~20% of cases) Drop to the full API when you need control that convenience methods don't expose: ```java theme={null} // Custom AgentCapabilities with specific MCP and prompt settings var caps = new AgentCapabilities( true, // loadSession new McpCapabilities(true, true), // HTTP + SSE new PromptCapabilities(true, false, true) // audio, embeddedContext, image ); return InitializeResponse.ok(caps); // Send non-text update types (Plan, ToolCall, AvailableCommandsUpdate, etc.) context.sendUpdate(sessionId, new Plan("plan", List.of( new PlanEntry("Analyze code", PlanEntryPriority.HIGH, PlanEntryStatus.IN_PROGRESS), new PlanEntry("Generate tests", PlanEntryPriority.MEDIUM, PlanEntryStatus.PENDING) ))); context.sendUpdate(sessionId, new ToolCall("tool_call", "search-1", "code-search", ToolKind.SEARCH, ToolCallStatus.IN_PROGRESS, null, null, null, null, null)); // Read file with offset and line limit var response = context.readTextFile( new ReadTextFileRequest(sessionId, "large-file.txt", 100, 50)); // Request permission with custom options var permResponse = context.requestPermission(new RequestPermissionRequest( sessionId, "Run deployment script?", List.of( new PermissionOption("allow-once", "Allow once", PermissionOptionKind.ALLOW_ONCE), new PermissionOption("always", "Always allow", PermissionOptionKind.ALLOW_ALWAYS), new PermissionOption("reject", "Reject", PermissionOptionKind.REJECT_ONCE) ))); ``` The convenience methods are wrappers around the full API — they call the same underlying protocol methods. You can mix and match freely within a single handler. *** ## Protocol Types All protocol types are defined in `AcpSchema` as Java records. ### Request/Response Types | Type | Fields | | :-------------------------------- | :--------------------------------------------------------------------------------------------------- | | `InitializeRequest` | `protocolVersion`, `clientCapabilities` | | `InitializeResponse` | `protocolVersion`, `agentCapabilities`, `authMethods`, `agentInfo` | | `AuthenticateRequest` | `methodId` | | `AuthenticateResponse` | *(empty)* | | `LogoutRequest` | *(empty)* *(0.14.0)* | | `LogoutResponse` | *(empty)* *(0.14.0)* | | `NewSessionRequest` | `cwd`, `mcpServers`, `additionalDirectories` *(`additionalDirectories` 0.14.0)* | | `NewSessionResponse` | `sessionId`, `modes`, ~~`models`~~ *(`models` deprecated — see below)* | | `LoadSessionRequest` | `sessionId`, `cwd`, `mcpServers`, `additionalDirectories` *(`additionalDirectories` 0.14.0)* | | `LoadSessionResponse` | `modes`, ~~`models`~~ *(deprecated)* | | `ListSessionsRequest` | `cwd` (optional filter), `cursor` (pagination) *(0.12.0)* | | `ListSessionsResponse` | `sessions` (list of `SessionInfo`), `nextCursor` *(0.12.0)* | | `ResumeSessionRequest` | `sessionId`, `cwd`, `mcpServers`, `additionalDirectories` *(0.12.0; `additionalDirectories` 0.14.0)* | | `ResumeSessionResponse` | `modes`, ~~`models`~~ *(0.12.0; `models` deprecated)* | | `CloseSessionRequest` | `sessionId` *(0.12.0)* | | `CloseSessionResponse` | *(empty)* *(0.12.0)* | | `DeleteSessionRequest` | `sessionId` *(0.14.0)* | | `DeleteSessionResponse` | *(empty)* *(0.14.0)* | | `ForkSessionRequest` | `sessionId`, `cwd`, `mcpServers`, `additionalDirectories` *(0.12.0, unstable)* | | `ForkSessionResponse` | `sessionId`, `modes`, ~~`models`~~, `configOptions` *(0.12.0, unstable; `models` deprecated)* | | `SetSessionConfigOptionRequest` | `sessionId`, `configId`, `value`, `type` *(0.12.0)* | | `SetSessionConfigOptionResponse` | `configOptions` (full config state) *(0.12.0)* | | `ListProvidersRequest` | *(empty)* *(0.14.0, unstable)* | | `ListProvidersResponse` | `providers` (list of `ProviderInfo`) *(0.14.0, unstable)* | | `SetProviderRequest` | `id`, `apiType`, `baseUrl`, `headers` *(0.14.0, unstable)* | | `SetProviderResponse` | *(empty)* *(0.14.0, unstable)* | | `DisableProviderRequest` | `id` *(0.14.0, unstable)* | | `DisableProviderResponse` | *(empty)* *(0.14.0, unstable)* | | `CreateElicitationRequest` | `sessionId`, `message`, `mode`, `requestedSchema` *(0.12.0, unstable)* | | `CreateElicitationResponse` | `action` (accept/decline/cancel), `content` *(0.12.0, unstable)* | | `CompleteElicitationNotification` | `elicitationId` *(0.12.0, unstable)* | | `PromptRequest` | `sessionId`, `prompt` (list of `ContentBlock`) | | `PromptResponse` | `stopReason` | | `CancelNotification` | `sessionId` | ### Session Types *(0.12.0)* | Type | Fields | Description | | :-------------------- | :------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------- | | `SessionInfo` | `sessionId`, `cwd`, `title`, `updatedAt`, `additionalDirectories` | Session metadata returned by `session/list` *(`additionalDirectories` 0.14.0)* | | `SessionCapabilities` | `list`, `close`, `resume`, `delete`, `additionalDirectories`, `fork` | Nested capability flags on `AgentCapabilities` *(`delete`, `additionalDirectories` 0.14.0; `fork` unstable)* | ### Config Option Types *(0.12.0)* `session/set_config_option` is stable. The `select` variant is the stable shape; `boolean` is an SDK extension. | Type | Description | | :-------------------------- | :--------------------------------------------------------------------- | | `SessionConfigOption` | Polymorphic: `SessionConfigSelect` or `SessionConfigBoolean` | | `SessionConfigSelect` | Select-type config with `currentValue`, `category`, and `options` list | | `SessionConfigBoolean` | Boolean toggle with `currentValue` *(unstable extension)* | | `SessionConfigSelectOption` | A named value within a select config (`value`, `name`) | | `ConfigOptionUpdate` | SessionUpdate variant for agent-pushed config changes | ### Provider Types *(0.14.0, unstable)* Model/backend routing configuration. The agent advertises support with a `providers` capability; the client manages providers via `listProviders` / `setProvider` / `disableProvider`. | Type | Description | | :---------------------- | :------------------------------------------------------------------------------- | | `ProviderInfo` | A configurable provider: `id`, `supported` (protocol ids), `required`, `current` | | `ProviderCurrentConfig` | Current effective routing: `apiType`, `baseUrl` | | `ProvidersCapabilities` | Agent capability marker (presence = supported), on `AgentCapabilities` | `apiType` / `supported` use well-known `LlmProtocol` ids (`anthropic`, `openai`, `azure`, `vertex`, `bedrock`) or a custom string, modeled as `String` in the SDK. ### Elicitation Types *(0.12.0, unstable)* | Type | Description | | :-------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------- | | `ElicitationSchema` | JSON Schema describing form fields (`properties`, `required`) | | `ElicitationPropertySchema` | Polymorphic: `StringPropertySchema`, `NumberPropertySchema`, `IntegerPropertySchema`, `BooleanPropertySchema`, `MultiSelectPropertySchema` | | `EnumOption` | Named value for select/multi-select (`const`, `title`) | | `ElicitationCapabilities` | Client capability for form and/or URL elicitation | | `ElicitationAction` | Enum: `ACCEPT`, `DECLINE`, `CANCEL` | ### Content Types | Type | Description | | :------------- | :----------------------------- | | `TextContent` | Text content with `text` field | | `ImageContent` | Image content (base64 or URL) | ### Session Update Types | Type | Description | | :--------------------------- | :-------------------------------------------------------- | | `UserMessageChunk` | Incremental user message text (optional `messageId`) | | `AgentMessageChunk` | Incremental response text (optional `messageId` — 0.14.0) | | `AgentThoughtChunk` | Agent thinking process (optional `messageId` — 0.14.0) | | `ToolCall` | Tool execution start | | `ToolCallUpdateNotification` | Tool progress update | | `Plan` | Agent's planned steps | | `AvailableCommandsUpdate` | Advertised slash commands | | `CurrentModeUpdate` | Agent mode change | | `UsageUpdate` | Context window and cost usage | | `ConfigOptionUpdate` | Session config option changes | Content chunks (`UserMessageChunk`, `AgentMessageChunk`, `AgentThoughtChunk`) carry an optional `messageId`: chunks sharing the same id belong to one logical message, and a change in id starts a new message *(0.14.0)*. ### Stop Reasons | Value | Description | | :----------- | :------------------------ | | `END_TURN` | Agent finished responding | | `MAX_TOKENS` | Token limit reached | | `REFUSAL` | Agent refused the request | | `CANCELLED` | Prompt was cancelled | ### Convenience Methods ```java theme={null} // Static factory methods InitializeResponse.ok() PromptResponse.endTurn() PromptResponse.text("response") ``` *** ## Capabilities ### Client Capabilities Advertised during `initialize`: ```java theme={null} client.initialize(new InitializeRequest(1, new ClientCapabilities( new FileSystemCapability(true, true), // read, write true // terminalExecution ))); ``` ### `NegotiatedCapabilities` Check capabilities before using them: ```java theme={null} NegotiatedCapabilities caps = context.getClientCapabilities(); if (caps.supportsReadTextFile()) { String content = context.readFile("file.txt"); } if (caps.supportsWriteTextFile()) { context.writeFile("output.txt", "content"); } ``` Or use `require` methods that throw `AcpCapabilityException` if unsupported: ```java theme={null} caps.requireWriteTextFile(); context.writeFile("output.txt", "content"); ``` ### Session Capabilities *(0.12.0)* Agents advertise session management support via `SessionCapabilities`. The `NegotiatedCapabilities` accessors are: `supportsListSessions()`, `supportsCloseSession()`, `supportsResumeSession()`, `supportsDeleteSession()` *(0.14.0)*, `supportsAdditionalDirectories()` *(0.14.0)*, and `supportsForkSession()` *(unstable)* — each with a matching `require*()` that throws `AcpCapabilityException`. ```java theme={null} NegotiatedCapabilities caps = client.getAgentCapabilities(); if (caps.supportsListSessions()) { client.listSessions(new ListSessionsRequest(null)); } if (caps.supportsDeleteSession()) { client.deleteSession(new DeleteSessionRequest(sessionId)); } if (caps.supportsAdditionalDirectories()) { client.newSession(new NewSessionRequest(cwd, List.of(), List.of("/extra/dir"))); } ``` ### Elicitation Capabilities *(0.12.0, unstable)* Clients advertise elicitation support during initialization: ```java theme={null} // Client: advertise form-mode elicitation support var caps = new ClientCapabilities( new FileSystemCapability(), false, new ElicitationCapabilities(), null); client.initialize(new InitializeRequest(1, caps)); ``` ```java theme={null} // Agent: check before sending elicitation if (context.getClientCapabilities().supportsElicitation()) { var response = context.createElicitation( CreateElicitationRequest.form(sessionId, "Pick one:", schema)); } ``` ### `@UnstableAcpApi` APIs marked `@UnstableAcpApi` correspond to protocol elements in `schema.unstable.json`. They are public and functional but may change in any minor release. When the protocol element stabilizes, the annotation is removed (compatible change). See [Versioning](#versioning) for the full policy. IntelliJ users can configure the *Unstable API Usage* inspection (*Settings > Inspections > JVM languages*) to flag usages. *** ## Transports | Transport | Client Class | Agent Class | Module | | ------------- | ----------------------------- | ---------------------------- | ------------------------------ | | **Stdio** | `StdioAcpClientTransport` | `StdioAcpAgentTransport` | acp-core | | **WebSocket** | `WebSocketAcpClientTransport` | `WebSocketAcpAgentTransport` | acp-core / acp-websocket-jetty | | **In-Memory** | via `InMemoryTransportPair` | via `InMemoryTransportPair` | acp-test | ### Stdio Transport The default transport. The client launches the agent as a subprocess and communicates via JSON-RPC over stdin/stdout. This is the same mechanism Zed, JetBrains, and VS Code use to talk to agents. **Client side** — `AgentParameters` specifies the command to launch. Any executable that speaks ACP over stdin/stdout works (Gemini CLI, your own agent JAR, etc.): ```java theme={null} var params = AgentParameters.builder("gemini") .arg("--experimental-acp") .build(); var transport = new StdioAcpClientTransport(params); ``` **Agent side** — reads JSON-RPC from stdin, writes responses to stdout. The agent doesn't need to know what launched it: ```java theme={null} var transport = new StdioAcpAgentTransport(); ``` ### WebSocket Transport For network-based communication. **Client (JDK-native, no extra dependencies):** ```java theme={null} var transport = new WebSocketAcpClientTransport( URI.create("ws://localhost:8080/acp"), AcpJsonMapper.createDefault() ); ``` **Agent (requires acp-websocket-jetty):** ```java theme={null} var transport = new WebSocketAcpAgentTransport( 8080, "/acp", AcpJsonMapper.createDefault() ); ``` ### In-Memory Transport For testing. No subprocess or network I/O. ```java theme={null} var pair = InMemoryTransportPair.create(); // pair.clientTransport() — for client // pair.agentTransport() — for agent // pair.closeGracefully() — cleanup ``` *** ## Errors ### Exception Hierarchy | Exception | Description | | :----------------------- | :-------------------------------------------- | | `AcpProtocolException` | JSON-RPC protocol error with code and message | | `AcpCapabilityException` | Tried to use an unsupported capability | | `AcpConnectionException` | Transport-level connection failure | ### Error Codes ```java theme={null} try { client.prompt(request); } catch (AcpProtocolException e) { if (e.isConcurrentPrompt()) { // Another prompt is already in progress } else if (e.isMethodNotFound()) { // Agent doesn't support this method } System.err.println("Error " + e.getCode() + ": " + e.getMessage()); } catch (AcpCapabilityException e) { System.err.println("Not supported: " + e.getCapability()); } ``` ### Agent-Side Error Handling Throw `AcpProtocolException` from handlers to send structured errors to clients: ```java theme={null} .promptHandler((req, context) -> { if (req.prompt().isEmpty()) { throw new AcpProtocolException( AcpErrorCodes.INVALID_PARAMS, "Empty prompt"); } // ... }) ``` *** ## Test Utilities The `acp-test` module provides utilities for testing without subprocesses. ### `InMemoryTransportPair` ```java theme={null} var pair = InMemoryTransportPair.create(); // Wire up agent AcpAsyncAgent agent = AcpAgent.async(pair.agentTransport()) .initializeHandler(req -> Mono.just(InitializeResponse.ok())) .newSessionHandler(req -> Mono.just( new NewSessionResponse(UUID.randomUUID().toString(), null, null))) .promptHandler((req, context) -> context.sendMessage("response") .then(Mono.just(PromptResponse.endTurn()))) .build(); agent.start().subscribe(); // Wire up client AcpSyncClient client = AcpClient.sync(pair.clientTransport()).build(); client.initialize(); // ... test ... pair.closeGracefully().block(); ``` *** ## Packages | Package | Description | | ------------------------------------------- | ------------------------------------------------------------- | | `com.agentclientprotocol.sdk.spec` | Protocol types (`AcpSchema.*`) | | `com.agentclientprotocol.sdk.client` | Client SDK (`AcpClient`, `AcpAsyncClient`, `AcpSyncClient`) | | `com.agentclientprotocol.sdk.agent` | Agent SDK (`AcpAgent`, `AcpAsyncAgent`, `AcpSyncAgent`) | | `com.agentclientprotocol.sdk.agent.support` | Annotation-based agent runtime (`AcpAgentSupport`) | | `com.agentclientprotocol.sdk.annotation` | Agent annotations (`@AcpAgent`, `@Prompt`, etc.) | | `com.agentclientprotocol.sdk.capabilities` | Capability negotiation (`NegotiatedCapabilities`) | | `com.agentclientprotocol.sdk.error` | Exceptions (`AcpProtocolException`, `AcpCapabilityException`) | | `com.agentclientprotocol.sdk.test` | Test utilities (`InMemoryTransportPair`) | ## Maven Artifacts | Artifact | Description | | --------------------- | -------------------------------------------------------------------- | | `acp-core` | Client and Agent SDKs, stdio and WebSocket client transports | | `acp-annotations` | `@AcpAgent`, `@Prompt`, and other annotations | | `acp-agent-support` | Annotation-based agent runtime (includes acp-annotations + acp-core) | | `acp-test` | In-memory transport and test utilities | | `acp-websocket-jetty` | Jetty-based WebSocket server transport for agents | *** ## See Also * [ACP Java SDK GitHub](https://github.com/agentclientprotocol/java-sdk) — Source code * [ACP Java Tutorial](https://github.com/markpollack/acp-java-tutorial) — 30 hands-on modules * [Agent Client Protocol](https://agentclientprotocol.com/) — Official specification # 01 first contact Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/01-first-contact # Module 01: First Contact Your first ACP client — launch an agent as a subprocess and send it a prompt. ## What You'll Learn * How ACP communication works (subprocess + stdin/stdout JSON-RPC) * Configuring agent process parameters with `AgentParameters` * Registering a `sessionUpdateConsumer` to see the agent's response * The three-phase lifecycle: initialize → newSession → prompt ## Prerequisites 1. **[Gemini CLI](https://github.com/google-gemini/gemini-cli) with ACP support** — the tutorial uses Gemini as a real ACP agent. Your client will launch it as a subprocess and talk to it over stdin/stdout. ```bash theme={null} gemini --experimental-acp --version ``` 2. **API key** ```bash theme={null} export GEMINI_API_KEY=your-key-here ``` 3. **Java 17 or later** ## The Code The client launches `gemini --experimental-acp` as a child process. `AgentParameters` builds the command line. `StdioAcpClientTransport` spawns the process and handles JSON-RPC message framing over its stdin/stdout. The `sessionUpdateConsumer` is how you see the agent's response. During `prompt()`, the agent streams back `AgentMessageChunk` updates containing the response text. Without a consumer, the prompt completes but you only get the stop reason — not the actual answer. From there, ACP follows a three-phase lifecycle: initialize the connection, create a session (with a working directory context), then send prompts. ```java theme={null} import com.agentclientprotocol.sdk.client.*; import com.agentclientprotocol.sdk.client.transport.*; import com.agentclientprotocol.sdk.spec.AcpSchema.*; import java.util.List; // 1. Build the command: "gemini --experimental-acp" var params = AgentParameters.builder("gemini") .arg("--experimental-acp") .build(); // 2. Launch it as a subprocess, communicate over stdin/stdout var transport = new StdioAcpClientTransport(params); // 3. Build client with update consumer to print the agent's response AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { if (notification.update() instanceof AgentMessageChunk msg) { System.out.print(((TextContent) msg.content()).text()); } }) .build(); // 4. Initialize — exchange protocol versions and capabilities client.initialize(); // 5. Create session — set working directory context var session = client.newSession( new NewSessionRequest(".", List.of())); // 6. Send prompt — blocks until agent responds, updates stream to consumer var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("What is 2+2? Reply with just the number.")) )); System.out.println("\nStop reason: " + response.stopReason()); client.close(); // Output: 4 // Stop reason: END_TURN ``` ## How It Works ACP communication follows a three-phase lifecycle: 1. **Initialize** — client and agent exchange protocol versions and capabilities 2. **New Session** — establishes a working directory context for the conversation 3. **Prompt** — sends content and receives a response with a stop reason The stdio transport is not Gemini-specific. Any executable that speaks ACP over stdin/stdout works — Gemini CLI, a custom agent JAR, or any other ACP-compliant tool. This is the same mechanism Zed, JetBrains, and VS Code use to talk to agents. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-01-first-contact) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-01-first-contact ``` ## Next Module [Module 05: Streaming Updates](/docs/acp-java-sdk/tutorial/05-streaming-updates) — receive real-time updates while the agent processes your prompt. Or skip to [Module 12: Echo Agent](/docs/acp-java-sdk/tutorial/12-echo-agent) to build your own agent (no API key required). # 02 protocol basics Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/02-protocol-basics # Module 02: Protocol Basics Deep dive into the ACP initialize handshake and version negotiation. ## What You'll Learn * The `InitializeRequest` and `InitializeResponse` structure * Protocol version negotiation semantics * Client and agent capability exchange ## How It Works The initialize handshake is the first message exchange in ACP — it must complete before any session or prompt operations. Both sides exchange: 1. **Protocol version** — they agree on a compatible version 2. **Client capabilities** — what the client can provide (file system access, terminal execution) 3. **Agent capabilities** — what the agent supports (session loading, image content, MCP) ## The Code Module 01 called `client.initialize()` with defaults. Here we use the explicit form to control exactly what capabilities we advertise. The `InitializeResponse` tells us what the agent supports: ```java theme={null} // Initialize with explicit protocol version and capabilities var initResponse = client.initialize( new InitializeRequest(1, new ClientCapabilities( new FileSystemCapability(true, true), // read, write false // terminalExecution ))); System.out.println("Protocol version: " + initResponse.protocolVersion()); System.out.println("Agent capabilities: " + initResponse.agentCapabilities()); System.out.println("Existing sessions: " + initResponse.sessionIds().size()); // Output: Protocol version: 1 // Agent capabilities: AgentCapabilities[...] // Existing sessions: 0 ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-02-protocol-basics) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-02-protocol-basics ``` ## Next Module [Module 03: Sessions](/docs/acp-java-sdk/tutorial/03-sessions) — create and manage conversation sessions. # 03 sessions Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/03-sessions # Module 03: Sessions Understanding session creation and lifecycle in ACP. ## What You'll Learn * Creating sessions with `NewSessionRequest` * Working directory context and context documents * Multiple independent sessions ## How Sessions Work Sessions are workspaces for conversations. Each session has a unique ID assigned by the agent, a working directory context, and its own conversation history. Multiple sessions on the same connection are fully independent. ## The Code After initializing (Module 01/02), the next step is creating a session. `NewSessionRequest` takes a working directory path and an optional list of MCP server configs. The agent returns a session ID that you use for all subsequent prompts: ```java theme={null} // Create a session with working directory context var session = client.newSession( new NewSessionRequest("/workspace", List.of())); System.out.println("Session ID: " + session.sessionId()); // Output: Session ID: 53d1a1ee-c7eb-4500-b25c-7bc2fdffa0e4 ``` You can create multiple sessions on the same connection. Each maintains its own conversation history — prompts in one session don't affect the other: ```java theme={null} var session1 = client.newSession(new NewSessionRequest("/project-a", List.of())); var session2 = client.newSession(new NewSessionRequest("/project-b", List.of())); // Prompts in session1 don't affect session2 client.prompt(new PromptRequest(session1.sessionId(), List.of(new TextContent("Analyze this project")))); ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-03-sessions) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-03-sessions ``` ## Next Module [Module 04: Prompts](/docs/acp-java-sdk/tutorial/04-prompts) — prompt requests and response handling in depth. # 04 prompts Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/04-prompts # Module 04: Prompts Deep dive into prompt requests and response handling. ## What You'll Learn * `PromptRequest` structure — session ID and content list * `PromptResponse` and `StopReason` values * Content types for prompts ## The Code `PromptRequest` takes a session ID (from Module 03) and a list of content items. The response includes a `StopReason` that tells you why the agent stopped generating — check this to know if the response is complete, truncated, or refused: ```java theme={null} // Send a prompt with text content var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Explain ACP in one sentence.")) )); System.out.println("Stop reason: " + response.stopReason()); // Output: Stop reason: END_TURN ``` ## Stop Reasons The `StopReason` tells you why the agent finished: | StopReason | Description | | ------------ | ---------------------------------- | | `END_TURN` | Agent finished responding normally | | `MAX_TOKENS` | Token limit reached | | `REFUSAL` | Agent refused the request | | `CANCELLED` | Prompt was cancelled by the client | Understanding stop reasons helps handle different agent behaviors. `END_TURN` is the normal case. `MAX_TOKENS` means the response was truncated. `REFUSAL` may require rephrasing the prompt. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-04-prompts) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-04-prompts ``` ## Next Module [Module 05: Streaming Updates](/docs/acp-java-sdk/tutorial/05-streaming-updates) — receive real-time updates during prompt processing. # 05 streaming updates Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/05-streaming-updates # Module 05: Streaming Updates Receive real-time updates from an agent while it processes your prompt. ## What You'll Learn * Registering a `sessionUpdateConsumer` on the client * Dispatching on `SessionUpdate` types with `instanceof` * Handling message chunks, thoughts, tool calls, and plans ## The Code The client registers an update consumer that receives each `SessionUpdate` as it arrives: ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { handleSessionUpdate(notification.update()); }) .build(); ``` The handler uses `instanceof` to dispatch on the `SessionUpdate` type: ```java theme={null} private static void handleSessionUpdate(SessionUpdate update) { if (update instanceof AgentMessageChunk msg) { System.out.print(((TextContent) msg.content()).text()); } else if (update instanceof AgentThoughtChunk thought) { System.out.println("[Thought] " + ((TextContent) thought.content()).text()); } else if (update instanceof ToolCall tool) { System.out.println("[Tool] " + tool.title() + " (" + tool.status() + ")"); } else if (update instanceof ToolCallUpdateNotification toolUpdate) { System.out.println("[Tool Update] " + toolUpdate.title() + " -> " + toolUpdate.status()); } else if (update instanceof Plan plan) { System.out.println("[Plan] " + plan.entries().size() + " steps"); } else if (update instanceof AvailableCommandsUpdate commands) { System.out.println("[Commands] " + commands.availableCommands().size() + " available"); } else if (update instanceof CurrentModeUpdate mode) { System.out.println("[Mode] " + mode.currentModeId()); } else if (update instanceof UsageUpdate usage) { System.out.println("[Usage] " + usage.used() + "/" + usage.size()); } } ``` ## Session Update Types | Type | Description | | ---------------------------- | ------------------------------------------------ | | `AgentMessageChunk` | Incremental response text (the main output) | | `AgentThoughtChunk` | Agent's thinking process | | `ToolCall` | Tool execution starting | | `ToolCallUpdateNotification` | Tool progress update | | `Plan` | Agent's planned steps with priorities and status | | `AvailableCommandsUpdate` | Slash commands the agent supports | | `CurrentModeUpdate` | Agent mode change | | `UsageUpdate` | Context window and cost usage | Updates arrive during `client.prompt()`. The prompt call blocks until the agent returns a `PromptResponse`, but updates stream in continuously through the consumer. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-05-streaming-updates) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-05-streaming-updates ``` ## Next Module [Module 12: Echo Agent](/docs/acp-java-sdk/tutorial/12-echo-agent) — build your first ACP agent (no API key required). # 06 update types Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/06-update-types # Module 06: Update Types Comprehensive coverage of all `SessionUpdate` types in ACP. ## What You'll Learn * All `SessionUpdate` types and when they appear * Dispatching on update types with `instanceof` * Building rich UIs that show agent activity ## The Code The client registers a `sessionUpdateConsumer` and uses `instanceof` to handle each type: ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { SessionUpdate update = notification.update(); if (update instanceof AgentMessageChunk msg) { System.out.print(((TextContent) msg.content()).text()); } else if (update instanceof AgentThoughtChunk thought) { System.out.println("[Thought] " + ((TextContent) thought.content()).text()); } else if (update instanceof ToolCall tc) { System.out.println("[Tool] " + tc.title() + " | " + tc.kind() + " | " + tc.status()); } else if (update instanceof ToolCallUpdateNotification tcUpdate) { System.out.println("[ToolUpdate] " + tcUpdate.toolCallId() + " -> " + tcUpdate.status()); } else if (update instanceof Plan plan) { System.out.println("[Plan] " + plan.entries().size() + " entries:"); plan.entries().forEach(entry -> System.out.println(" - " + entry.content() + " [" + entry.status() + "]")); } else if (update instanceof AvailableCommandsUpdate commands) { System.out.println("[Commands] " + commands.availableCommands().size() + " available"); } else if (update instanceof CurrentModeUpdate mode) { System.out.println("[Mode] " + mode.currentModeId()); } else if (update instanceof UsageUpdate usage) { System.out.printf("[Usage] %d/%d tokens%n", usage.used(), usage.size()); if (usage.cost() != null) { System.out.printf(" Cost: %.4f %s%n", usage.cost().amount(), usage.cost().currency()); } } }) .build(); ``` ## Update Types Reference | Type | Content | Typical Use | | ---------------------------- | ------------------------- | --------------------------------------------- | | `AgentMessageChunk` | Incremental response text | Main output, streamed word by word | | `AgentThoughtChunk` | Agent's thinking process | Show reasoning in a collapsible panel | | `ToolCall` | Tool execution start | Show tool name, kind, and status | | `ToolCallUpdateNotification` | Tool progress | Update status of in-progress tool | | `Plan` | Agent's planned steps | Show step list with priorities and completion | | `AvailableCommandsUpdate` | Slash commands | Populate command palette | | `CurrentModeUpdate` | Mode change | Update UI mode indicator | | `UsageUpdate` | Token usage and cost | Show context window usage bar | This module extends Module 05 by handling every update type rather than just messages. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-06-update-types) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-06-update-types ``` ## Next Module [Module 07: Agent Requests](/docs/acp-java-sdk/tutorial/07-agent-requests) — handle file read/write requests from agents. # 07 agent requests Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/07-agent-requests # Module 07: Agent Requests (Client Side) Handle file read/write requests from agents on the client side. ## What You'll Learn * Registering `readTextFileHandler` and `writeTextFileHandler` * Advertising file system capabilities via `ClientCapabilities` * The inverted request flow: agents request, clients serve ## Inverted Request Flow In ACP, the request direction is inverted for file operations compared to traditional client-server: the **agent** requests files from the **client**. This allows agents to access the user's local filesystem through a controlled interface — the client decides which files to expose and how to handle writes. ## The Code To enable this, the client registers file handlers on its builder and advertises file system capabilities during initialize. When the agent calls `context.readFile()` or `context.writeFile()`, these handlers are invoked: ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .readTextFileHandler(req -> { Path path = Path.of(req.path()); if (!Files.exists(path)) { throw new RuntimeException("File not found: " + req.path()); } return new ReadTextFileResponse(Files.readString(path)); }) .writeTextFileHandler(req -> { Files.writeString(Path.of(req.path()), req.content()); return new WriteTextFileResponse(); }) .sessionUpdateConsumer(notification -> { /* handle updates */ }) .build(); // Advertise file system capabilities client.initialize(new InitializeRequest(1, new ClientCapabilities( new FileSystemCapability(true, true), // read=true, write=true false // terminalExecution ))); ``` Throw exceptions from handlers for errors. The SDK converts exceptions to JSON-RPC error responses. Do not return error strings as content. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-07-agent-requests) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-07-agent-requests ``` ## Next Module [Module 08: Permissions](/docs/acp-java-sdk/tutorial/08-permissions) — handle permission requests from agents. # 08 permissions Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/08-permissions # Module 08: Permissions Handle permission requests from agents on the client side. ## What You'll Learn * Registering a `requestPermissionHandler` * `PermissionOption` and `PermissionOptionKind` types * Permission outcomes: selected vs cancelled ## The Code When an agent wants to perform a sensitive operation (like writing a file or running a command), it can ask the client for permission first. The client registers a `requestPermissionHandler` that receives the request details and a list of options (allow once, allow always, reject). Your handler decides which option to select — typically by showing a dialog to the user: ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .requestPermissionHandler(req -> { System.out.println("Agent wants to: " + req.toolCall().title()); System.out.println("Options:"); for (var option : req.options()) { System.out.println(" " + option.id() + ": " + option.name() + " (" + option.kind() + ")"); } // Auto-approve with first option return new RequestPermissionResponse( req.options().getFirst().id()); }) .readTextFileHandler(req -> /* file handler */) .writeTextFileHandler(req -> /* file handler */) .build(); ``` ## Permission Option Kinds | Kind | Description | | --------------- | ----------------------------- | | `ALLOW_ONCE` | Allow this specific operation | | `ALLOW_ALWAYS` | Allow all similar operations | | `REJECT_ONCE` | Deny this specific operation | | `REJECT_ALWAYS` | Deny all similar operations | The agent sends a `RequestPermissionRequest` containing the tool call details and a list of options. The client presents choices to the user and returns the selected option ID. This completes the bidirectional request flow (with Module 07 for files). ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-08-permissions) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-08-permissions ``` ## Next Module [Module 09: Session Resume](/docs/acp-java-sdk/tutorial/09-session-resume) — load and resume existing sessions. # 09 session resume Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/09-session-resume # Module 09: Session Resume Load and resume existing sessions. ## What You'll Learn * Using `client.loadSession()` to resume a session by ID * `LoadSessionRequest` and `LoadSessionResponse` structure * Session state persistence across load operations ## How It Works `loadSession()` tells the agent to resume a previously created session. The agent looks up its stored state for that session ID and restores the conversation context. Whether state actually persists depends on the agent implementation — the demo includes a `StatefulAgent` that stores session history in a `ConcurrentHashMap`. ## The Code This example creates a session, sends messages to build history, then loads the same session by ID. After loading, the agent has access to the previous conversation context: ```java theme={null} // Create a session and send some messages var session = client.newSession(new NewSessionRequest(".", List.of())); client.prompt(new PromptRequest(session.sessionId(), List.of(new TextContent("Remember: my favorite color is blue.")))); // Later: resume the same session by ID var loadResponse = client.loadSession( new LoadSessionRequest(session.sessionId(), ".", List.of())); // Continue the conversation with context preserved client.prompt(new PromptRequest(session.sessionId(), List.of(new TextContent("What is my favorite color?")))); // The agent remembers the previous conversation ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-09-session-resume) ## Running the Example ```bash theme={null} ./mvnw package -pl module-09-session-resume -q ./mvnw exec:java -pl module-09-session-resume ``` ## Next Module [Module 10: Cancellation](/docs/acp-java-sdk/tutorial/10-cancellation) — cancel an in-progress prompt. # 10 cancellation Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/10-cancellation # Module 10: Cancellation Cancel an in-progress prompt from the client side. ## What You'll Learn * Sending `CancelNotification` to interrupt a running prompt * Running prompts in background threads * How cancellation affects `StopReason` ## The Code ```java theme={null} // Run prompt in a background thread AtomicReference responseRef = new AtomicReference<>(); CompletableFuture promptFuture = CompletableFuture.runAsync(() -> { var response = client.prompt(new PromptRequest( sessionId, List.of(new TextContent("Do a long task")))); responseRef.set(response); }); // Wait, then cancel Thread.sleep(1500); client.cancel(new CancelNotification(sessionId)); // Wait for prompt to finish promptFuture.join(); System.out.println("Stop reason: " + responseRef.get().stopReason()); ``` ## How It Works `client.cancel()` sends a one-way notification (not a request) to the agent. The agent's `cancelHandler` receives it and sets a flag. The prompt handler checks this flag between steps and stops early when cancelled. On the agent side: ```java theme={null} // Track cancellation per session Map cancelledSessions = new ConcurrentHashMap<>(); AcpSyncAgent agent = AcpAgent.sync(transport) .cancelHandler(notification -> { // Set flag — no response needed (notification, not request) cancelledSessions.put(notification.sessionId(), true); }) .promptHandler((req, context) -> { cancelledSessions.put(req.sessionId(), false); for (int i = 1; i <= 10; i++) { // Check flag before each step if (cancelledSessions.getOrDefault(req.sessionId(), false)) { context.sendMessage("[Cancelled at step " + i + "]"); return PromptResponse.endTurn(); } context.sendMessage("Step " + i + "/10... "); Thread.sleep(500); } context.sendMessage("All steps completed!"); return PromptResponse.endTurn(); }) .build(); ``` Cancellation is cooperative. The agent must check for it — the SDK does not forcefully interrupt handler execution. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-10-cancellation) ## Running the Example ```bash theme={null} ./mvnw package -pl module-10-cancellation -q ./mvnw exec:java -pl module-10-cancellation ``` ## Next Module [Module 11: Error Handling](/docs/acp-java-sdk/tutorial/11-error-handling) — handle protocol errors from agents. # 11 error handling Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/11-error-handling # Module 11: Error Handling Handle protocol errors from agents on the client side. ## What You'll Learn * Catching `AcpClientSession.AcpError` * Standard error codes in `AcpErrorCodes` * Throwing `AcpProtocolException` from agent handlers * Error recovery — continuing after errors ## The Code ACP uses structured errors based on JSON-RPC error codes. On the **client side**, protocol errors arrive as `AcpClientSession.AcpError` exceptions. You can inspect the error code to determine what went wrong: ```java theme={null} // Client: catch protocol errors try { client.prompt(new PromptRequest(sessionId, List.of(new TextContent("this is invalid input")))); } catch (AcpClientSession.AcpError e) { System.out.println("Code: " + e.getCode()); System.out.println("Message: " + e.getMessage()); // Output: Code: -32602 // Message: Invalid parameter in prompt: 'this is invalid input' } ``` On the **agent side**, throw `AcpProtocolException` with a standard error code. The SDK converts it to a JSON-RPC error response: ```java theme={null} // Agent: throw protocol errors .promptHandler((req, context) -> { String text = /* extract text from prompt */; if (text.contains("invalid")) { throw new AcpProtocolException( AcpErrorCodes.INVALID_PARAMS, "Invalid parameter in prompt: '" + text + "'"); } if (text.contains("internal")) { throw new AcpProtocolException( AcpErrorCodes.INTERNAL_ERROR, "Simulated internal error"); } context.sendMessage("Success! Processed: " + text); return PromptResponse.endTurn(); }) ``` ## Error Codes | Code | Constant | When to Use | | -------- | ------------------- | ------------------------ | | `-32602` | `INVALID_PARAMS` | Bad input from client | | `-32603` | `INTERNAL_ERROR` | Unexpected agent failure | | `-32001` | `SESSION_NOT_FOUND` | Unknown session ID | | `-32002` | `PERMISSION_DENIED` | Client lacks permission | Agents throw `AcpProtocolException` with one of these codes. The SDK converts it to a JSON-RPC error response. Clients catch it as `AcpClientSession.AcpError`. ## Error Recovery Errors do not terminate the connection. After catching an error, the client can continue sending requests on the same session: ```java theme={null} // This works — errors don't break the connection try { client.prompt(/* bad input */); } catch (AcpClientSession.AcpError e) { // handle error } // Continue normally var response = client.prompt(/* good input */); // Works fine ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-11-error-handling) ## Running the Example ```bash theme={null} ./mvnw package -pl module-11-error-handling -q ./mvnw exec:java -pl module-11-error-handling ``` ## Next Module [Module 12: Echo Agent](/docs/acp-java-sdk/tutorial/12-echo-agent) — build your first agent. # 12 echo agent Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/12-echo-agent # Module 12: Echo Agent Build a minimal ACP agent in \~25 lines. No API key required. ## What You'll Learn * Building an agent with `AcpAgent.sync()` * Implementing the three required handlers: initialize, newSession, prompt * Sending messages back to the client * Running a self-contained agent + client demo ## The Agent ```java theme={null} import com.agentclientprotocol.sdk.agent.*; import com.agentclientprotocol.sdk.agent.transport.*; import com.agentclientprotocol.sdk.spec.AcpSchema.*; import java.util.UUID; var transport = new StdioAcpAgentTransport(); AcpSyncAgent agent = AcpAgent.sync(transport) .initializeHandler(req -> InitializeResponse.ok()) .newSessionHandler(req -> new NewSessionResponse(UUID.randomUUID().toString(), null, null)) .promptHandler((req, context) -> { // Echo it back using convenience method context.sendMessage("Echo: " + req.text()); return PromptResponse.endTurn(); }) .build(); agent.run(); // Blocks until client disconnects ``` ## How It Works An ACP agent needs three handlers: | Handler | Purpose | | ------------------- | ------------------------------------------------------- | | `initializeHandler` | Protocol handshake — return capabilities | | `newSessionHandler` | Create a session — return a unique session ID | | `promptHandler` | Process prompts — send updates and return a stop reason | `agent.run()` starts the agent and blocks. The agent reads JSON-RPC requests from stdin and writes responses to stdout. This is the stdio transport — the same mechanism Zed and JetBrains use to talk to agents. The `context` parameter in the prompt handler gives access to `sendMessage()`, `sendThought()`, and other convenience methods for sending updates back to the client. ## The Demo Client The module also includes `EchoAgentDemo.java`, which launches the echo agent as a subprocess and exercises it: ```java theme={null} // Launch echo agent as subprocess (from packaged JAR) var params = AgentParameters.builder("java") .arg("-jar") .arg(jarPath) .build(); var transport = new StdioAcpClientTransport(params); AcpSyncClient client = AcpClient.sync(transport) .sessionUpdateConsumer(notification -> { if (notification.update() instanceof AgentMessageChunk msg) { System.out.println(((TextContent) msg.content()).text()); } }) .build(); client.initialize(); var session = client.newSession(new NewSessionRequest(".", List.of())); client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Hello, Echo Agent!")) )); // Output: Echo: Hello, Echo Agent! ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-12-echo-agent) ## Running the Example ```bash theme={null} ./mvnw package -pl module-12-echo-agent -q ./mvnw exec:java -pl module-12-echo-agent ``` ## Key Points * **No API key** — the agent runs entirely locally * **Stdio transport** — same protocol mechanism used by Zed, JetBrains, VS Code * **`agent.run()`** — combines `start()` and `awaitTermination()` * **`context.sendMessage()`** — convenience for sending `AgentMessageChunk` updates ## Next Module [Module 13: Agent Handlers](/docs/acp-java-sdk/tutorial/13-agent-handlers) — implement all handler types including load session and cancel. # 13 agent handlers Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/13-agent-handlers # Module 13: Agent Handlers Implement all handler types that an ACP agent can provide. ## What You'll Learn * All five handler types: initialize, newSession, loadSession, prompt, cancel * Session tracking with `ConcurrentHashMap` * Logging to stderr (stdout is reserved for the protocol) ## The Code An ACP agent has five handler types. Three are required (`initialize`, `newSession`, `prompt`) and two are optional (`loadSession`, `cancel`). This example shows all five wired up with the sync builder API. Note that `stdout` is reserved for the JSON-RPC protocol — agent logging goes to `stderr`: ```java theme={null} import com.agentclientprotocol.sdk.agent.*; import com.agentclientprotocol.sdk.agent.transport.*; import com.agentclientprotocol.sdk.spec.AcpSchema.*; import java.util.*; import java.util.concurrent.ConcurrentHashMap; var sessions = new ConcurrentHashMap(); var transport = new StdioAcpAgentTransport(); AcpSyncAgent agent = AcpAgent.sync(transport) .initializeHandler(req -> { System.err.println("Protocol version: " + req.protocolVersion()); return InitializeResponse.ok(); }) .newSessionHandler(req -> { String sessionId = UUID.randomUUID().toString(); sessions.put(sessionId, req.cwd()); System.err.println("New session: " + sessionId); return new NewSessionResponse(sessionId, null, null); }) .loadSessionHandler(req -> { System.err.println("Loading session: " + req.sessionId()); return new LoadSessionResponse(List.of(), List.of()); }) .promptHandler((req, context) -> { context.sendMessage("Received: " + req.text()); return PromptResponse.endTurn(); }) .cancelHandler(notification -> { System.err.println("Cancel requested for: " + notification.sessionId()); }) .build(); agent.run(); ``` ## Handler Reference | Handler | Method | Required | Description | | -------------------- | ---------------- | -------- | -------------------------------------------------- | | `initializeHandler` | `initialize` | Yes | Protocol handshake, capability exchange | | `newSessionHandler` | `session/new` | Yes | Create session with working directory | | `loadSessionHandler` | `session/load` | No | Resume an existing session by ID | | `promptHandler` | `session/prompt` | Yes | Process user prompts | | `cancelHandler` | `session/cancel` | No | Handle cancellation (fire-and-forget notification) | The `cancelHandler` receives a `CancelNotification`, not a request — it has no response. This is the JSON-RPC notification pattern: the client sends it and does not expect a reply. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-13-agent-handlers) ## Running the Example ```bash theme={null} ./mvnw package -pl module-13-agent-handlers -q ./mvnw exec:java -pl module-13-agent-handlers ``` ## Next Module [Module 14: Sending Updates](/docs/acp-java-sdk/tutorial/14-sending-updates) — send all types of session updates to clients. # 14 sending updates Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/14-sending-updates # Module 14: Sending Updates Send all types of session updates from an agent to its client. ## What You'll Learn * Convenience methods: `sendMessage()`, `sendThought()` * Full API: `sendUpdate()` for complex types (plans, tool calls, commands) * All `SessionUpdate` types from the agent's perspective ## The Code The prompt handler demonstrates each update type: ```java theme={null} .promptHandler((req, context) -> { String sessionId = context.getSessionId(); // 1. Thought — show thinking process (convenience method) context.sendThought("Let me analyze this request..."); // 2. Plan — show steps and progress (full API) context.sendUpdate(sessionId, new Plan("plan", List.of( new PlanEntry("Analyze the prompt", PlanEntryPriority.HIGH, PlanEntryStatus.IN_PROGRESS), new PlanEntry("Generate response", PlanEntryPriority.HIGH, PlanEntryStatus.PENDING), new PlanEntry("Format output", PlanEntryPriority.MEDIUM, PlanEntryStatus.PENDING) ))); // 3. Tool Call — show tool execution starting context.sendUpdate(sessionId, new ToolCall("tool_call", "tool-1", "Analyzing prompt", ToolKind.THINK, ToolCallStatus.IN_PROGRESS, List.of(), null, null, null, null)); // 4. Tool Call Update — show progress context.sendUpdate(sessionId, new ToolCallUpdateNotification("tool_call_update", "tool-1", "Analyzing prompt", ToolKind.THINK, ToolCallStatus.COMPLETED, List.of(), null, null, null, null)); // 5. Available Commands — advertise slash commands context.sendUpdate(sessionId, new AvailableCommandsUpdate("available_commands_update", List.of( new AvailableCommand("help", "Show help", new AvailableCommandInput("topic")), new AvailableCommand("clear", "Clear context", null) ))); // 6. Mode Update — report current mode context.sendUpdate(sessionId, new CurrentModeUpdate("current_mode_update", "default")); // 7. Usage Update — report token usage and cost context.sendUpdate(sessionId, new UsageUpdate("usage_update", 53000L, 200000L)); // 8. Message chunks — the actual response (convenience method) context.sendMessage("Here is my response "); context.sendMessage("streamed in "); context.sendMessage("multiple chunks."); return PromptResponse.endTurn(); }) ``` ## Convenience vs Full API | Method | Sends | When to Use | | --------------------------------------- | ------------------- | ---------------------------------- | | `context.sendMessage(text)` | `AgentMessageChunk` | Response text | | `context.sendThought(text)` | `AgentThoughtChunk` | Thinking process | | `context.sendUpdate(sessionId, update)` | Any `SessionUpdate` | Plans, tool calls, commands, modes | Convenience methods handle wrapping in `TextContent` and setting the `type` field. Use `sendUpdate()` for complex types that need full control over their structure. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-14-sending-updates) ## Running the Example ```bash theme={null} ./mvnw package -pl module-14-sending-updates -q ./mvnw exec:java -pl module-14-sending-updates ``` ## Next Module [Module 15: Agent Requests](/docs/acp-java-sdk/tutorial/15-agent-requests) — read files, write files, and request permissions from the client. # 15 agent requests Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/15-agent-requests # Module 15: Agent Requests Agents can request file operations and permissions from the client. ## What You'll Learn * Reading files from the client with `context.readFile()` * Writing files with `context.writeFile()` * Requesting permissions with `context.requestPermission()` * The client-side handler pattern for responding to agent requests ## The Agent The prompt handler uses `SyncPromptContext` convenience methods: ```java theme={null} .promptHandler((req, context) -> { String sessionId = context.getSessionId(); // 1. Read a file (convenience method) context.sendMessage("Reading pom.xml from your system...\n"); String content = context.readFile("pom.xml", 0, 10); context.sendMessage("File content (first 10 lines):\n" + content + "\n\n"); // 2. Request permission (full API for complex permissions) ToolCallUpdate toolCall = new ToolCallUpdate( "tool-write-1", "Create summary.txt", ToolKind.EDIT, ToolCallStatus.PENDING, null, null, null, null ); List options = List.of( new PermissionOption("allow", "Allow this once", PermissionOptionKind.ALLOW_ONCE), new PermissionOption("allow_always", "Always allow", PermissionOptionKind.ALLOW_ALWAYS), new PermissionOption("deny", "Deny", PermissionOptionKind.REJECT_ONCE) ); var permissionResponse = context.requestPermission( new RequestPermissionRequest(sessionId, toolCall, options)); context.sendMessage("Permission: " + permissionResponse.outcome() + "\n"); // 3. Write a file (convenience method) context.writeFile("summary.txt", "Created by the FileRequestingAgent.\n"); context.sendMessage("Successfully wrote summary.txt!\n"); return PromptResponse.endTurn(); }) ``` ## The Client The client registers handlers to respond to agent requests: ```java theme={null} AcpSyncClient client = AcpClient.sync(transport) .readTextFileHandler(req -> { String fileContent = Files.readString(Path.of(req.path())); return new ReadTextFileResponse(fileContent); }) .writeTextFileHandler(req -> { Files.writeString(Path.of(req.path()), req.content()); return new WriteTextFileResponse(); }) .requestPermissionHandler(req -> { // Auto-approve with first option return new RequestPermissionResponse( req.options().getFirst().id()); }) .build(); ``` Throw exceptions from handlers for errors. The SDK converts exceptions to JSON-RPC error responses. Do not return error strings as content — agents will misinterpret them as file content. ## Client Capabilities The client must advertise file system support during initialization: ```java theme={null} client.initialize(new InitializeRequest(1, new ClientCapabilities( new FileSystemCapability(true, true), // read=true, write=true false // terminalExecution ))); ``` Agents can check capabilities before using them: ```java theme={null} NegotiatedCapabilities caps = context.getClientCapabilities(); if (caps.supportsReadTextFile()) { String content = context.readFile("file.txt"); } ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-15-agent-requests) ## Running the Example ```bash theme={null} ./mvnw package -pl module-15-agent-requests -q ./mvnw exec:java -pl module-15-agent-requests ``` ## Next Module [Module 16: In-Memory Testing](/docs/acp-java-sdk/tutorial/16-in-memory-testing) — test agents without subprocess launching. # 16 in memory testing Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/16-in-memory-testing # Module 16: In-Memory Testing Test client-agent communication without subprocesses or I/O. ## What You'll Learn * Using `InMemoryTransportPair` from `acp-test` * Wiring a client and agent together in-process * Fast, deterministic testing without external dependencies ## The Code `InMemoryTransportPair` from the `acp-test` module creates a pair of connected transports — one for the client, one for the agent. Messages pass through in-memory buffers instead of subprocess stdin/stdout. This makes tests fast, deterministic, and free of external dependencies. The pattern below wires up an agent and client in the same JVM, sends a prompt, and verifies the round-trip: ```java theme={null} import com.agentclientprotocol.sdk.test.InMemoryTransportPair; import com.agentclientprotocol.sdk.agent.*; import com.agentclientprotocol.sdk.client.*; import com.agentclientprotocol.sdk.spec.AcpSchema.*; import reactor.core.publisher.Mono; import java.util.*; import java.util.concurrent.atomic.AtomicReference; // 1. Create in-memory transport pair var transportPair = InMemoryTransportPair.create(); // 2. Create agent (using async API here) AtomicReference receivedPrompt = new AtomicReference<>(); AcpAsyncAgent agent = AcpAgent.async(transportPair.agentTransport()) .initializeHandler(req -> Mono.just(InitializeResponse.ok())) .newSessionHandler(req -> Mono.just(new NewSessionResponse( UUID.randomUUID().toString(), null, null))) .promptHandler((req, context) -> { String text = req.text(); receivedPrompt.set(text); return context.sendMessage("Echo: " + text) .then(Mono.just(PromptResponse.endTurn())); }) .build(); agent.start().subscribe(); // 3. Create client AtomicReference receivedMessage = new AtomicReference<>(); AcpSyncClient client = AcpClient.sync(transportPair.clientTransport()) .sessionUpdateConsumer(notification -> { if (notification.update() instanceof AgentMessageChunk msg) { receivedMessage.set(((TextContent) msg.content()).text()); } }) .build(); // 4. Run test client.initialize(); var session = client.newSession(new NewSessionRequest(".", List.of())); client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Hello from in-memory test!")) )); // 5. Verify assert "Hello from in-memory test!".equals(receivedPrompt.get()); assert "Echo: Hello from in-memory test!".equals(receivedMessage.get()); // 6. Cleanup client.close(); transportPair.closeGracefully().block(); ``` ## How It Works `InMemoryTransportPair.create()` returns a pair of connected transports: | Transport | Used By | Description | | --------------------------------- | ------- | ------------------------------------- | | `transportPair.clientTransport()` | Client | Sends to agent, receives from agent | | `transportPair.agentTransport()` | Agent | Receives from client, sends to client | Messages pass through in-memory buffers. No subprocess launching, no stdin/stdout, no network I/O. This makes tests fast and deterministic. ## Maven Dependency ```xml theme={null} com.agentclientprotocol acp-test 0.15.0 test ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-16-in-memory-testing) ## Running the Example ```bash theme={null} ./mvnw exec:java -pl module-16-in-memory-testing ``` ## Next Module [Module 17: Capability Negotiation](/docs/acp-java-sdk/tutorial/17-capability-negotiation) — advertise and check capabilities between client and agent. # 17 capability negotiation Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/17-capability-negotiation # Module 17: Capability Negotiation Client and agent agree on what each supports during initialization. ## What You'll Learn * Advertising `ClientCapabilities` during `initialize()` * Checking `NegotiatedCapabilities` from the agent side * Graceful degradation when a capability is missing ## The Code ### Client: Advertise capabilities ```java theme={null} // Tell the agent what we support var clientCaps = new ClientCapabilities( new FileSystemCapability(true, true), // readTextFile, writeTextFile true // terminal ); client.initialize(new InitializeRequest(1, clientCaps)); // Check what the agent supports NegotiatedCapabilities agentCaps = client.getAgentCapabilities(); System.out.println("loadSession: " + agentCaps.supportsLoadSession()); System.out.println("mcpHttp: " + agentCaps.supportsMcpHttp()); System.out.println("mcpSse: " + agentCaps.supportsMcpSse()); ``` ### Agent: Advertise and check capabilities ```java theme={null} .initializeHandler(req -> { // Read what the client supports var clientCaps = req.clientCapabilities(); // Advertise our own capabilities var agentCaps = new AgentCapabilities( true, // loadSession new McpCapabilities(false, false), // no MCP new PromptCapabilities(false, false, true) // embeddedContext only ); return InitializeResponse.ok(agentCaps); }) .promptHandler((req, context) -> { // Check capabilities before attempting operations NegotiatedCapabilities caps = context.getClientCapabilities(); if (caps.supportsReadTextFile()) { String content = context.readFile("/etc/hostname"); } else { // Graceful degradation context.sendMessage("File read not supported by client"); } return PromptResponse.endTurn(); }) ``` ## Capability Categories | Capability | Client | Agent | | ---------------------- | --------------------------- | ------------------------------------------- | | `FileSystemCapability` | readTextFile, writeTextFile | — | | Terminal | terminal execution | — | | `AgentCapabilities` | — | loadSession | | `McpCapabilities` | — | HTTP, SSE | | `PromptCapabilities` | — | imageContent, audioContent, embeddedContext | Clients advertise file system and terminal support. Agents advertise session resume, MCP server types, and content format support. Both sides can call `getClientCapabilities()` or `getAgentCapabilities()` after initialization to check what was negotiated. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-17-capability-negotiation) ## Running the Example ```bash theme={null} ./mvnw package -pl module-17-capability-negotiation -q ./mvnw exec:java -pl module-17-capability-negotiation ``` ## Next Module [Module 18: Terminal Operations](/docs/acp-java-sdk/tutorial/18-terminal-operations) — execute shell commands through the terminal API. # 18 terminal operations Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/18-terminal-operations # Module 18: Terminal Operations Execute shell commands on the client through the terminal API. ## What You'll Learn * The four-step terminal lifecycle: create, wait, output, release * Implementing terminal handlers on the client * Using the terminal API from the agent side ## The Code ### Client: Implement terminal handlers ```java theme={null} var clientCaps = new ClientCapabilities( new FileSystemCapability(false, false), true // terminal enabled ); AcpSyncClient client = AcpClient.sync(transport) .createTerminalHandler(req -> { List cmd = new ArrayList<>(); cmd.add(req.command()); if (req.args() != null) cmd.addAll(req.args()); Process process = new ProcessBuilder(cmd) .redirectErrorStream(true) .start(); terminals.put(terminalId, process); return new CreateTerminalResponse(terminalId); }) .waitForTerminalExitHandler(req -> { Process process = terminals.get(req.terminalId()).process(); int exitCode = process.waitFor(); return new WaitForTerminalExitResponse(exitCode, null); }) .terminalOutputHandler(req -> { String output = capturedOutput.get(req.terminalId()); return new TerminalOutputResponse(output, false, null); }) .releaseTerminalHandler(req -> { Process process = terminals.remove(req.terminalId()).process(); process.destroyForcibly(); return new ReleaseTerminalResponse(); }) .build(); client.initialize(new InitializeRequest(1, clientCaps)); ``` ### Agent: Use terminal API ```java theme={null} .promptHandler((req, context) -> { // Check capability first if (!context.getClientCapabilities().supportsTerminal()) { context.sendMessage("Terminal not supported"); return PromptResponse.endTurn(); } String terminalId = null; try { // Step 1: Create terminal var createResp = context.createTerminal( new CreateTerminalRequest( context.getSessionId(), "sh", List.of("-c", command), null, null, null)); terminalId = createResp.terminalId(); // Step 2: Wait for exit var exitResp = context.waitForTerminalExit( new WaitForTerminalExitRequest(context.getSessionId(), terminalId)); // Step 3: Get output var outputResp = context.getTerminalOutput( new TerminalOutputRequest(context.getSessionId(), terminalId)); context.sendMessage("Exit: " + exitResp.exitCode() + "\nOutput:\n" + outputResp.output()); } finally { // Step 4: Always release if (terminalId != null) { context.releaseTerminal( new ReleaseTerminalRequest(context.getSessionId(), terminalId)); } } return PromptResponse.endTurn(); }) ``` ## Terminal Lifecycle | Step | Agent calls | Client handles | Purpose | | ---- | ----------------------- | ---------------------------- | ------------------ | | 1 | `createTerminal()` | `createTerminalHandler` | Spawn process | | 2 | `waitForTerminalExit()` | `waitForTerminalExitHandler` | Block until done | | 3 | `getTerminalOutput()` | `terminalOutputHandler` | Read stdout/stderr | | 4 | `releaseTerminal()` | `releaseTerminalHandler` | Clean up resources | The agent requests command execution, but the client controls what actually runs. This keeps command execution under the user's control — the client decides whether to allow, sandbox, or deny terminal requests. The SDK also provides `context.execute()` as a convenience method that combines all four steps. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-18-terminal-operations) ## Running the Example ```bash theme={null} ./mvnw package -pl module-18-terminal-operations -q ./mvnw exec:java -pl module-18-terminal-operations ``` ## Next Module [Module 19: MCP Servers](/docs/acp-java-sdk/tutorial/19-mcp-servers) — pass MCP server configurations to agents. # 19 mcp servers Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/19-mcp-servers # Module 19: MCP Servers Pass MCP server configurations to agents when creating sessions. ## What You'll Learn * Passing `McpServer` configs via `NewSessionRequest` * Three MCP server types: `McpServerStdio`, `McpServerHttp`, `McpServerSse` * Checking agent MCP capabilities * Receiving MCP configs on the agent side ## The Code ### Client: Pass MCP servers to agent ```java theme={null} // Check what MCP transports the agent supports NegotiatedCapabilities agentCaps = client.getAgentCapabilities(); System.out.println("HTTP: " + agentCaps.supportsMcpHttp()); System.out.println("SSE: " + agentCaps.supportsMcpSse()); // Session with STDIO MCP server var session1 = client.newSession(new NewSessionRequest(cwd, List.of( new McpServerStdio( "filesystem", "npx", List.of("-y", "@modelcontextprotocol/server-filesystem", "/tmp"), List.of()) ))); // Session with multiple server types var session2 = client.newSession(new NewSessionRequest(cwd, List.of( new McpServerStdio("git", "npx", List.of("-y", "@modelcontextprotocol/server-git"), List.of()), new McpServerHttp("weather-api", "https://api.weather.example.com/mcp", List.of()), new McpServerSse("live-data", "https://stream.example.com/mcp/events", List.of()) ))); ``` ### Agent: Receive and advertise MCP support ```java theme={null} .initializeHandler(req -> { var mcpCaps = new McpCapabilities(true, true); // HTTP and SSE var agentCaps = new AgentCapabilities( true, mcpCaps, new PromptCapabilities()); return InitializeResponse.ok(agentCaps); }) .newSessionHandler(req -> { // MCP servers arrive with the session List servers = req.mcpServers(); for (McpServer server : servers) { switch (server) { case McpServerStdio s -> System.out.println("STDIO: " + s.name() + " " + s.command()); case McpServerHttp h -> System.out.println("HTTP: " + h.name() + " " + h.url()); case McpServerSse s -> System.out.println("SSE: " + s.name() + " " + s.url()); default -> {} } } return new NewSessionResponse(sessionId, null, null); }) ``` ## MCP Server Types | Type | Transport | Use Case | | ---------------- | ----------------------- | ----------------------------- | | `McpServerStdio` | Stdin/stdout subprocess | Local tools (filesystem, git) | | `McpServerHttp` | HTTP endpoint | Remote APIs | | `McpServerSse` | Server-sent events | Streaming data | The client tells the agent which MCP servers are available. The agent is responsible for connecting to them using an MCP client library. ACP handles the configuration exchange — not the MCP connection itself. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-19-mcp-servers) ## Running the Example ```bash theme={null} ./mvnw package -pl module-19-mcp-servers -q ./mvnw exec:java -pl module-19-mcp-servers ``` ## Next Module [Module 21: Async Client](/docs/acp-java-sdk/tutorial/21-async-client) — use the reactive, non-blocking client API. # 21 async client Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/21-async-client # Module 21: Async Client The reactive, non-blocking version of Module 01. ## What You'll Learn * `AcpClient.async()` instead of `AcpClient.sync()` * Chaining operations with `Mono` and `flatMap` * Async session update consumers returning `Mono` * When to use async vs sync ## The Code ```java theme={null} // Build async client — note AcpAsyncClient return type AcpAsyncClient client = AcpClient.async(transport) .sessionUpdateConsumer(notification -> { // Async consumer must return Mono var update = notification.update(); if (update instanceof AgentMessageChunk msg) { if (msg.content() instanceof TextContent text) { System.out.print(text.text()); } } return Mono.empty(); }) .build(); // Chain operations reactively with flatMap client.initialize() .flatMap(init -> client.newSession( new NewSessionRequest(".", List.of()))) .flatMap(session -> client.prompt( new PromptRequest(session.sessionId(), List.of(new TextContent("What is 2+2?"))))) .flatMap(response -> client.closeGracefully()) .subscribe( unused -> {}, error -> System.err.println("Error: " + error.getMessage()), () -> System.out.println("Done!") ); ``` ## Sync vs Async Comparison | Aspect | `AcpClient.sync()` | `AcpClient.async()` | | --------------- | --------------------- | ------------------------------ | | Return type | `T` | `Mono` | | Chaining | Sequential statements | `flatMap` | | Update consumer | `void` | `Mono` | | Blocking | Yes | No (unless you call `block()`) | The async client wraps every operation in Project Reactor's `Mono`. If you're already using a reactive framework, the async client integrates naturally. For CLI tools and scripts, the sync client is simpler. ## Alternative: block() For scripts where you want async types but don't care about non-blocking I/O: ```java theme={null} var init = client.initialize().block(); var session = client.newSession(new NewSessionRequest(".", List.of())).block(); var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Hello")))).block(); client.closeGracefully().block(); ``` This defeats the purpose of async but can be useful during prototyping. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-21-async-client) ## Running the Example ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw compile exec:java -pl module-21-async-client ``` ## Next Module [Module 22: Async Agent](/docs/acp-java-sdk/tutorial/22-async-agent) — build an agent with reactive handlers. # 22 async agent Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/22-async-agent # Module 22: Async Agent The reactive, non-blocking version of Module 12 (Echo Agent). ## What You'll Learn * `AcpAgent.async()` instead of `AcpAgent.sync()` * Handlers returning `Mono` instead of `T` * Chaining `sendMessage()` with `then()` before returning the response * Agent lifecycle: `start().block()` + `awaitTermination().block()` ## The Code ```java theme={null} AcpAsyncAgent agent = AcpAgent.async(transport) // Returns Mono .initializeHandler(req -> Mono.just(InitializeResponse.ok())) // Returns Mono .newSessionHandler(req -> Mono.just(new NewSessionResponse( UUID.randomUUID().toString(), null, null))) // Returns Mono .promptHandler((req, context) -> { String text = req.text(); // sendMessage() returns Mono — must chain with then() return context.sendMessage("Async Echo: " + text) .then(Mono.just(PromptResponse.endTurn())); }) .build(); // Start and block until transport closes agent.start().block(); agent.awaitTermination().block(); ``` ## Sync vs Async Agent Comparison | Aspect | `AcpAgent.sync()` | `AcpAgent.async()` | | --------------- | ----------------- | ------------------------------------------------------ | | Handler return | `T` | `Mono` | | `sendMessage()` | `void` (blocking) | `Mono` (must chain) | | Lifecycle | `agent.run()` | `agent.start().block()` + `awaitTermination().block()` | The critical difference: in the async agent, `context.sendMessage()` returns `Mono`. You must chain it with `.then()` before returning the response `Mono`. If you skip the chain, the message won't be sent before the response. ```java theme={null} // Wrong — message may not be sent before response context.sendMessage("text"); return Mono.just(PromptResponse.endTurn()); // Right — message is sent, then response follows return context.sendMessage("text") .then(Mono.just(PromptResponse.endTurn())); ``` ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-22-async-agent) ## Running the Example ```bash theme={null} ./mvnw package -pl module-22-async-agent -q ./mvnw exec:java -pl module-22-async-agent ``` ## Key Points * **No API key** — the async agent runs entirely locally, same as Module 12 * **Same protocol** — sync and async agents are interchangeable from the client's perspective * **Reactive integration** — async agents work naturally with Project Reactor, R2DBC, and other reactive libraries ## Previous Module [Module 21: Async Client](/docs/acp-java-sdk/tutorial/21-async-client) — the reactive client API. # 23 spring boot agent Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/23-spring-boot-agent # Module 23: Spring Boot Agent Build an ACP agent as a Spring Boot application. No manual transport or lifecycle wiring required. ## Prerequisites * Java 21+ (Spring Boot 4.x requirement) * Completed [Module 12: Echo Agent](/docs/acp-java-sdk/tutorial/12-echo-agent) ## What You'll Learn * Using `@AcpAgent` annotations with Spring Boot autoconfiguration * How the starter eliminates boilerplate transport and lifecycle code * Redirecting logging to stderr for stdio agents ## Dependencies Add the ACP Spring Boot Starter: ```xml theme={null} org.springaicommunity acp-spring-boot-starter 0.11.1 ``` ## The Agent Compare this with [Module 12's builder-based agent](/docs/acp-java-sdk/tutorial/12-echo-agent). The annotation approach replaces the builder chain with annotated methods on a Spring bean: ```java theme={null} @Component @AcpAgent(name = "echo-agent", version = "1.0") public class EchoAgentBean { @Initialize public InitializeResponse initialize(InitializeRequest request) { return InitializeResponse.ok(); } @NewSession public NewSessionResponse newSession(NewSessionRequest request) { return new NewSessionResponse(UUID.randomUUID().toString(), null, null); } @Prompt public PromptResponse prompt(PromptRequest request, SyncPromptContext context) { context.sendMessage("Echo: " + request.text()); return PromptResponse.endTurn(); } } ``` The application class is a standard `@SpringBootApplication`: ```java theme={null} @SpringBootApplication public class EchoAgentApplication { public static void main(String[] args) { SpringApplication.run(EchoAgentApplication.class, args); } } ``` ## What the Autoconfiguration Does When Spring Boot starts, the ACP autoconfiguration: 1. **Creates a `StdioAcpAgentTransport`** — the default for agents (reads stdin, writes stdout) 2. **Discovers the `@AcpAgent` bean** — scans the application context for exactly one `@AcpAgent`-annotated bean 3. **Wires through `AcpAgentSupport`** — resolves `@Initialize`, `@NewSession`, `@Prompt` handler methods 4. **Starts via `SmartLifecycle`** — the agent starts after the application context refreshes and stops on shutdown No explicit `agent.run()` call. No manual transport creation. Spring manages it all. ## Stdio and Logging Agent stdout is reserved for the JSON-RPC protocol. Spring Boot's default logging writes to stdout, which would corrupt the protocol stream. Three configuration changes fix this: **application.properties:** ```properties theme={null} # Disable banner — stdout is reserved for JSON-RPC spring.main.banner-mode=off # Keep the JVM alive (no web server to block) spring.main.keep-alive=true ``` The `keep-alive` setting is essential. Without it, the Spring Boot application starts the agent, then exits immediately because there's no web server keeping the JVM alive. **logback-spring.xml:** ```xml theme={null} System.err %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n ``` ## Build & Run ```bash theme={null} # Package the Spring Boot agent ./mvnw package -pl module-23-spring-boot-agent -q # Run the demo (launches agent as subprocess and talks to it) ./mvnw exec:java -pl module-23-spring-boot-agent ``` ## Module 12 vs Module 23 | Aspect | Module 12 (Builder) | Module 23 (Spring Boot) | | ------------- | ------------------------------------- | ---------------------------- | | Transport | Manual `new StdioAcpAgentTransport()` | Autoconfigured | | Handlers | Lambda callbacks via builder | Annotated methods on a bean | | Lifecycle | Explicit `agent.run()` | `SmartLifecycle` (automatic) | | Configuration | Hardcoded in Java | `application.properties` | | Dependencies | `acp-core` only | `acp-spring-boot-starter` | ## Configuration Properties | Property | Default | Description | | ---------------------------------- | ------- | --------------------------------------- | | `spring.acp.agent.enabled` | `true` | Enable/disable agent autoconfiguration | | `spring.acp.agent.request-timeout` | `60s` | Request processing timeout | | `spring.acp.agent.transport.type` | `stdio` | Transport type (currently only `stdio`) | ## Next [Module 24: Spring Boot Client](/docs/acp-java-sdk/tutorial/24-spring-boot-client) — use the autoconfigured client to connect to agents. [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-23-spring-boot-agent) # 24 spring boot client Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/24-spring-boot-client # Module 24: Spring Boot Client Use the autoconfigured ACP client in a Spring Boot application. Connect to agents with just properties — no manual transport or client construction. ## Prerequisites * Java 21+ (Spring Boot 4.x requirement) * [Module 23: Spring Boot Agent](/docs/acp-java-sdk/tutorial/23-spring-boot-agent) built and available ## What You'll Learn * Injecting `AcpSyncClient` from autoconfiguration * Configuring the client transport via `application.properties` * Property-driven transport selection (stdio vs WebSocket) ## Dependencies Same starter as the agent side: ```xml theme={null} org.springaicommunity acp-spring-boot-starter 0.11.1 ``` ## The Client The autoconfigured `AcpSyncClient` is injected like any Spring bean: ```java theme={null} @SpringBootApplication public class ClientApplication { public static void main(String[] args) { SpringApplication.run(ClientApplication.class, args); } @Bean CommandLineRunner demo(AcpSyncClient client) { return args -> { // Initialize the connection client.initialize(); // Create a session String cwd = System.getProperty("user.dir"); var session = client.newSession(new NewSessionRequest(cwd, List.of())); // Send a prompt var response = client.prompt(new PromptRequest( session.sessionId(), List.of(new TextContent("Hello from Spring Boot client!")))); System.out.println("Stop reason: " + response.stopReason()); }; } } ``` ## Configuration **application.properties:** ```properties theme={null} # Connect to the module-23 agent via stdio transport spring.acp.client.transport.stdio.command=java spring.acp.client.transport.stdio.args=-jar,module-23-spring-boot-agent/target/module-23-spring-boot-agent-1.0.0-SNAPSHOT.jar # Increase timeout for Spring Boot agent startup spring.acp.client.request-timeout=60s ``` The autoconfiguration detects the `stdio.command` property and creates a `StdioAcpClientTransport` that launches the agent as a subprocess. ## What the Autoconfiguration Does 1. **Detects transport properties** — `stdio.command` triggers stdio transport; `websocket.uri` triggers WebSocket 2. **Creates `AcpSyncClient` and `AcpAsyncClient`** — configured with timeout and client capabilities 3. **Manages shutdown** — `DisposableBean` calls `closeGracefully()` on context close ## Transport Selection The autoconfiguration picks the transport based on which properties are set: | Properties Set | Transport Created | | -------------------------------------------- | ----------------------------- | | `spring.acp.client.transport.stdio.command` | `StdioAcpClientTransport` | | `spring.acp.client.transport.websocket.uri` | `WebSocketAcpClientTransport` | | `spring.acp.client.transport.type=stdio` | Explicit stdio selection | | `spring.acp.client.transport.type=websocket` | Explicit WebSocket selection | ## Build & Run ```bash theme={null} # Build both the agent and client ./mvnw package -pl module-23-spring-boot-agent,module-24-spring-boot-client -q # Run the client (from repo root) ./mvnw spring-boot:run -pl module-24-spring-boot-client ``` ## Client Configuration Properties | Property | Default | Description | | ------------------------------------------------------- | ----------- | ------------------------------- | | `spring.acp.client.request-timeout` | `30s` | Request timeout | | `spring.acp.client.transport.type` | auto-detect | `stdio` or `websocket` | | `spring.acp.client.transport.stdio.command` | — | Command to launch agent | | `spring.acp.client.transport.stdio.args` | — | Command arguments | | `spring.acp.client.transport.stdio.env.*` | — | Environment variables | | `spring.acp.client.transport.websocket.uri` | — | WebSocket URI | | `spring.acp.client.transport.websocket.connect-timeout` | `10s` | Connection timeout | | `spring.acp.client.capabilities.read-text-file` | `true` | Advertise file read capability | | `spring.acp.client.capabilities.write-text-file` | `true` | Advertise file write capability | | `spring.acp.client.capabilities.terminal` | `false` | Advertise terminal capability | [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-24-spring-boot-client) # 28 zed integration Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/28-zed-integration # Module 28: Zed Integration Run your Java ACP agent inside the Zed editor. ## What You'll Learn * Building an agent JAR for editor integration * Configuring Zed to launch your agent * The stdio transport mechanism editors use to communicate with agents ## Prerequisites * [Zed editor](https://zed.dev/download) installed * Java 17+ * Agent JAR built (see below) ## The Agent The agent is the same sync builder pattern from Module 12, with conversational responses: ```java theme={null} var transport = new StdioAcpAgentTransport(); AcpSyncAgent agent = AcpAgent.sync(transport) .initializeHandler(req -> { System.err.println("[ZedAgent] Received initialize request"); return InitializeResponse.ok(); }) .newSessionHandler(req -> { System.err.println("[ZedAgent] Creating session for cwd: " + req.cwd()); return new NewSessionResponse( UUID.randomUUID().toString(), null, null); }) .promptHandler((req, context) -> { String promptText = req.text(); context.sendThought("Processing your request..."); String response = generateResponse(promptText); context.sendMessage(response); return PromptResponse.endTurn(); }) .build(); agent.run(); ``` Logging goes to stderr because Zed captures stdout for the JSON-RPC protocol. ## Build and Configure ### 1. Build the JAR ```bash theme={null} ./mvnw package -pl module-28-zed-integration -q ``` ### 2. Get the absolute path ```bash theme={null} realpath module-28-zed-integration/target/zed-agent.jar ``` ### 3. Configure Zed Open Zed settings (`Ctrl+,` on Linux, `Cmd+,` on Mac) and add: ```json theme={null} { "agent_servers": { "Java Tutorial Agent": { "type": "custom", "command": "java", "args": ["-jar", "/absolute/path/to/zed-agent.jar"] } } } ``` ### 4. Use in Zed Open the Agent Panel (`Ctrl+?`), click **+**, select **Java Tutorial Agent**, and start chatting. ## How It Works Zed launches your agent as a subprocess and communicates via JSON-RPC over stdio. This is the same mechanism used by LSP language servers. Your agent receives `initialize`, `session/new`, and `session/prompt` requests, and responds with JSON-RPC responses and notifications. No code changes are needed for different editors. The same JAR works in Zed, JetBrains, and VS Code. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-28-zed-integration) ## Running the Example ```bash theme={null} ./mvnw package -pl module-28-zed-integration -q ./mvnw exec:java -pl module-28-zed-integration ``` ## Next Module [Module 29: JetBrains Integration](/docs/acp-java-sdk/tutorial/29-jetbrains-integration) — configure the same agent for IntelliJ, PyCharm, and other JetBrains IDEs. # 29 jetbrains integration Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/29-jetbrains-integration # Module 29: JetBrains Integration Run your Java ACP agent in IntelliJ IDEA, PyCharm, WebStorm, and other JetBrains IDEs. ## What You'll Learn * Configuring JetBrains IDEs for ACP agents * The `~/.jetbrains/acp.json` configuration format * MCP server access from JetBrains ## Prerequisites * JetBrains IDE version 25.3 RC or later * JetBrains AI Assistant plugin enabled * Java 17+ ## The Agent Same code as Module 28 (Zed). The agent is identical — only the IDE configuration differs: ```java theme={null} AcpSyncAgent agent = AcpAgent.sync(transport) .initializeHandler(req -> InitializeResponse.ok()) .newSessionHandler(req -> new NewSessionResponse(UUID.randomUUID().toString(), null, null)) .promptHandler((req, context) -> { String promptText = /* extract from req */; context.sendThought("Analyzing your request..."); context.sendMessage(generateResponse(promptText)); return PromptResponse.endTurn(); }) .build(); agent.run(); ``` ## Build and Configure ### 1. Build the JAR ```bash theme={null} ./mvnw package -pl module-29-jetbrains-integration -q ``` ### 2. Configure JetBrains Create or edit `~/.jetbrains/acp.json`: ```json theme={null} { "agent_servers": { "Java Tutorial Agent": { "command": "java", "args": ["-jar", "/absolute/path/to/jetbrains-agent.jar"] } } } ``` Or use the IDE: AI Chat tool window → gear icon → Configure ACP Agents. ### 3. Use the agent Open AI Chat (`Alt+Shift+C`), select **Java Tutorial Agent** from the agent dropdown. ## Configuration Options ### With environment variables ```json theme={null} { "agent_servers": { "Java Tutorial Agent": { "command": "java", "args": ["-jar", "/path/to/jetbrains-agent.jar"], "env": { "MY_API_KEY": "your-key-here" } } } } ``` ### With IDE MCP server access ```json theme={null} { "agent_servers": { "Java Tutorial Agent": { "command": "java", "args": ["-jar", "/path/to/jetbrains-agent.jar"], "use_idea_mcp": true, "use_custom_mcp": true } } } ``` ## Supported IDEs All JetBrains IDEs with AI Assistant support ACP in version 25.3 RC and later: IntelliJ IDEA, PyCharm, WebStorm, GoLand, PhpStorm, Rider, CLion, RubyMine, DataGrip. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-29-jetbrains-integration) ## Running the Example ```bash theme={null} ./mvnw package -pl module-29-jetbrains-integration -q ./mvnw exec:java -pl module-29-jetbrains-integration ``` ## Next Module [Module 30: VS Code Integration](/docs/acp-java-sdk/tutorial/30-vscode-integration) — connect to VS Code using the community extension. # 30 vscode integration Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/30-vscode-integration # Module 30: VS Code Integration Run your Java ACP agent in VS Code using the community vscode-acp extension. ## What You'll Learn * Installing the vscode-acp extension * Creating a PATH-discoverable wrapper script * The differences between VS Code, Zed, and JetBrains ACP configuration ## Prerequisites * VS Code installed * Java 17+ ## The Agent Same code as Modules 28-29. No code changes needed for different editors. ## Build and Configure ### 1. Install the vscode-acp extension ```bash theme={null} code --install-extension omercnet.vscode-acp ``` Or search "VSCode ACP" in the Extensions panel (`Ctrl+Shift+X`). ### 2. Build the JAR ```bash theme={null} ./mvnw package -pl module-30-vscode-integration -q ``` ### 3. Create a wrapper script The extension auto-detects agents from your PATH. Create a wrapper: **Linux/macOS:** ```bash theme={null} cat > ~/.local/bin/java-tutorial-agent << 'EOF' #!/bin/bash exec java -jar /absolute/path/to/vscode-agent.jar "$@" EOF chmod +x ~/.local/bin/java-tutorial-agent ``` Replace `/absolute/path/to/vscode-agent.jar` with the output of: ```bash theme={null} realpath module-30-vscode-integration/target/vscode-agent.jar ``` **Windows:** Create `%USERPROFILE%\bin\java-tutorial-agent.cmd`: ```cmd theme={null} @echo off java -jar C:\path\to\vscode-agent.jar %* ``` ### 4. Use in VS Code Click the **VSCode ACP** icon in the Activity Bar, click **Connect**, select your agent. ## Configuration Comparison | IDE | Configuration | Discovery | | ------------- | ----------------------------------------------- | ----------------------- | | **Zed** | `settings.json` — direct command + args | Explicit in settings | | **JetBrains** | `~/.jetbrains/acp.json` — direct command + args | Explicit in config file | | **VS Code** | PATH wrapper script | Auto-detected from PATH | The same agent JAR works across all three editors. The only difference is how each editor finds and launches the agent. ## Native VS Code Support VS Code does not have native ACP support yet. Microsoft is tracking it in [Issue #265496](https://github.com/microsoft/vscode/issues/265496). The community extension provides ACP functionality until native support arrives. ## Source Code [View on GitHub](https://github.com/markpollack/acp-java-tutorial/tree/main/module-30-vscode-integration) ## Running the Example ```bash theme={null} ./mvnw package -pl module-30-vscode-integration -q ./mvnw exec:java -pl module-30-vscode-integration ``` # Tutorial Source: https://lab.pollack.ai/docs/acp-java-sdk/tutorial/index A progressive tutorial for learning the ACP Java SDK — from client basics to IDE integration. A progressive, hands-on tutorial. Each module focuses on one concept and includes runnable source code. ## Prerequisites * Java 17 or later * Maven 3.8+ (or use the included `./mvnw` wrapper) * For client modules (01-11, 21): [Gemini CLI](https://github.com/google-gemini/gemini-cli) with `--experimental-acp` flag, and a `GEMINI_API_KEY`. The tutorial uses Gemini as a real ACP agent to talk to — the SDK launches it as a subprocess and communicates over stdin/stdout. * For agent modules (12-19, 22): no external dependencies. You build the agent and the tutorial provides a test client that launches it. ## Tutorial Structure | Part | Modules | Topics | | ---------------------- | ------- | -------------------------------------------------------------------------------------------------- | | **1. Client Basics** | 01-11 | Connect, sessions, prompts, streaming, updates, file handlers, permissions, resume, cancel, errors | | **2. Building Agents** | 12-19 | Echo agent, handlers, updates, requests, testing, capabilities, terminal, MCP | | **3. Advanced** | 21-22 | Async client, async agent (Project Reactor) | | **4. IDE Integration** | 28-30 | Zed, JetBrains, VS Code | ## Getting the Code ```bash theme={null} git clone https://github.com/markpollack/acp-java-tutorial.git cd acp-java-tutorial ./mvnw compile ``` ## Running a Module Agent modules run locally with no API key: ```bash theme={null} ./mvnw package -pl module-12-echo-agent -q ./mvnw exec:java -pl module-12-echo-agent ``` Client modules require `GEMINI_API_KEY`: ```bash theme={null} export GEMINI_API_KEY=your-key-here ./mvnw exec:java -pl module-01-first-contact ``` ## Three Agent API Styles The SDK provides three ways to build agents: | Style | Entry Point | Programming Model | | -------------------- | ---------------------- | -------------------------------------- | | **Annotation-based** | `@AcpAgent`, `@Prompt` | Declarative, least boilerplate | | **Sync** | `AcpAgent.sync()` | Blocking handlers, plain return values | | **Async** | `AcpAgent.async()` | Reactive, Project Reactor `Mono` | The tutorial uses Sync for agent examples (most accessible to most developers) and introduces annotations and async where relevant. ## Start Learning Begin with [Module 01: First Contact](/docs/acp-java-sdk/tutorial/01-first-contact) to connect to your first ACP agent. Or jump to [Module 12: Echo Agent](/docs/acp-java-sdk/tutorial/12-echo-agent) to build an agent without any API key. # What's New Source: https://lab.pollack.ai/docs/acp-java-sdk/whats-new Release notes for ACP Java SDK — auto-generated from git history ## 0.15.0 (2026-08-21) Correctness and supply-chain hygiene. No protocol or public API changes — a drop-in upgrade from 0.14.0. * Fix non-deterministic notification ordering — notifications are serialized through a sink drained by `concatMap`, preserving arrival order (thanks @ljiro) * Drain queued notifications on graceful close instead of discarding them; `closeGracefully()` now waits for the drain, bounded by the session `requestTimeout` * Distinguish expected from anomalous notification-sink emission failures in logging * Security: Jackson 2.21.2 → 2.21.5 and Jetty 12.0.14 → 12.0.37, clearing 17 known advisories (5 high) in the published dependency closure * Ship the verbatim Apache License 2.0, and package LICENSE and NOTICE under `META-INF` in every artifact * Run the integration tests — three `*IT` classes existed but were never executed by `mvn verify` * Pin GitHub Actions to commit SHAs; bound workflow jobs to 30 minutes ## 0.14.0 (2026-06-11) * Docs: 0.14.0 release notes in README and CHANGELOG * Add unstable providers/\* methods * Add messageId overloads to prompt context send methods * Promote session config-option API to stable * Deprecate the session-model API for removal * Add logout and session/delete agent methods * Add additionalDirectories and per-chunk messageId fields * Add WebSocket edge case tests, increase max message size to 4MB * Fix WebSocket client transport echoing agent requests back to agent * Update version references to 0.12.0, add snapshot instructions * Bump version to 0.13.0-SNAPSHOT ## 0.12.0 (2026-05-25) * Add tutorial cross-references for module-20 and module-31 in release notes * Add elicitation decline and cancel integration tests * Update README with versioning policy and 0.12.0-SNAPSHOT release notes * Add session/fork and session/set\_config\_option (unstable) * Add elicitation/create and elicitation/complete (unstable) * Add @UnstableAcpApi stability marker annotation * Add session/list, session/close, session/resume methods * Bump version to 0.12.0-SNAPSHOT ## 0.11.0 (2026-05-12) * Remove MCP SDK dependency, add own JSON abstraction layer * Update .gitignore with local tool config * Bump version to 0.11.0-SNAPSHOT ## 0.10.0 (2026-04-10) * Restore Duration import removed by PR #1 * Add Implementation type with clientInfo/agentInfo fields * Add UsageUpdate support to SessionUpdate types * Add support for UsageUpdate in SessionUpdate interface and related tests * Update README for 0.9.0 release * Bump version to 0.10.0-SNAPSHOT * style: remove unused imports ## 0.9.0 (2026-03-31) * Initial release. # Agent Configuration Source: https://lab.pollack.ai/docs/agent-bench/agent-config Configure a local CLI process as an Agent Bench agent ## Agent YAML Format This page describes the agent process contract in the 0.6.0 release. ```yaml theme={null} command: timeout: ``` Agent Bench passes the command to `bash -c` with the task workspace as the working directory. The command should read `INSTRUCTION.md`, change the workspace, and exit. It runs as a local host process with the invoking user's permissions. The workspace directory is an organizational boundary, not a security boundary. Agent Bench does not provide container execution or isolation. Use your own disposable VM, CI runner, or other externally managed isolation for untrusted agents or benchmark definitions. Benchmark `setup` and `post` commands have the same host authority as the configured agent command. ## Examples ```yaml theme={null} # Claude Code command: claude --print --dangerously-skip-permissions "Read INSTRUCTION.md and follow the instructions precisely." timeout: PT45M ``` ```yaml theme={null} # Gemini CLI command: gemini -p "Read INSTRUCTION.md and follow the instructions." timeout: PT30M ``` ```yaml theme={null} # Local script command: ./my-agent.sh timeout: PT10M ``` ```bash theme={null} #!/bin/bash instruction=$(cat INSTRUCTION.md) # Use the instruction in your agent logic. printf '%s\n' 'Hello World!' > hello.txt ``` ## Filesystem Contract After `provide`, the workspace contains: | File | Description | | --------------------- | -------------------------------------------------- | | `INSTRUCTION.md` | Task description | | `.bench-context.yaml` | Benchmark, task, version, and timeout context | | Source files | Workspace-template files when supplied by the task | The external agent may create or modify files in this directory. Agent Judge implementations inspect that result during grading. ## Optional Agent Journal If the agent writes `journal.yaml`, Agent Bench records efficiency metrics with the trial: ```yaml theme={null} schema: bench.journal.v1 totalTurns: 8 totalInputTokens: 4000 totalOutputTokens: 2000 totalCostUsd: 0.12 durationMs: 15000 ``` Agents without a journal are still graded; only the journal-derived efficiency fields are absent. ## Optional Trajectory Reference If the agent writes `trajectory-ref.txt` containing a path or URI, Agent Bench copies the reference into the trial result for later analysis. Agent Bench does not upload or manage the referenced trace. # CLI Reference Source: https://lab.pollack.ai/docs/agent-bench/cli-reference Agent Bench core and agents-module entry points, commands, and flags ## Entry Points The 0.6.0 release publishes both CLI entry points. Run them from a `v0.6.0` source checkout: ```bash theme={null} # BenchMain: deterministic core CLI ./mvnw -q -pl agent-bench-core exec:java -Dexec.args=" [flags]" # BenchApp: core commands plus the real Agent Client-backed LLM judge for run/resume ./mvnw -q -pl agent-bench-agents exec:java -Dexec.args=" [flags]" ``` `BenchMain` registers Agent Judge's deterministic file, build, and coverage judges. During `run` and `resume`, its `test-quality-llm` registration abstains because no model-backed judge is present. `BenchApp` replaces that registration with `TestQualityJudge`, backed by Agent Client and the Claude agent implementation. Other commands delegate to the core CLI. All benchmark setup, post-processing, and configured agent commands execute as local host processes with the invoking user's permissions. Workspaces do not provide isolation. ## Discovery Commands ### `list` ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java -Dexec.args="list" ``` ### `tasks` ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="tasks --benchmark hello-world" ``` `--benchmark` is required. ## End-to-End Commands ### `run` Prepares each selected task, runs setup scripts and the configured agent, runs post-processing, and grades the result. ```bash theme={null} ./mvnw -q -pl agent-bench-agents exec:java \ -Dexec.args="run --benchmark code-coverage --agent agents/claude-code.yaml" ``` | Flag | Required | Description | | -------------- | -------- | -------------------------------------------------------- | | `--benchmark` | Yes | Benchmark name | | `--agent` | No | Agent config YAML; omission creates a manual-mode result | | `--task` | No | Run one task ID | | `--difficulty` | No | Filter tasks by `easy`, `medium`, or `hard` | ### `resume` Resumes `runs/`, preserving completed trial results and running missing trials. ```bash theme={null} ./mvnw -q -pl agent-bench-agents exec:java \ -Dexec.args="resume --run-id 386aba4a-4285-45d2-bbf4-a159c00c3f3b" ``` `--run-id` is required. ### `compare` ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="compare --runs runs/ runs/" ``` `--runs` accepts one or more run-directory paths and prints accuracy, pass\@k, cost, duration, and trial-count comparisons. ## Split Workflow ### `provide` Creates the workspace, copies an optional template, and writes `INSTRUCTION.md` and `.bench-context.yaml`. ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="provide --benchmark hello-world --task hello-world --workspace /tmp/ws" ``` ### `grade` Evaluates an existing workspace with the benchmark's deterministic Agent Judge configuration. ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="grade --benchmark hello-world --task hello-world --workspace /tmp/ws" ``` Both commands require `--benchmark` and `--workspace`; `--task` is optional only for a single-task benchmark. The built-in split CLI is for deterministic juries. Use agents-module `run` or `resume` when the jury includes the real `test-quality-llm` judge. ## Output Structure ```text theme={null} runs// result.json run-metadata.json bench.lock tasks/ / result.json workspace/ ``` Trial failures use the result model's `FailureMode` values, including agent timeout/error, setup or grade errors, build/test failures, context exhaustion, and unknown failures. # Getting Started with Agent Bench Source: https://lab.pollack.ai/docs/agent-bench/getting-started Run a benchmark end-to-end, split the agent step, or use the Java API ## Prerequisites * Java 21 * An AI coding agent with a CLI for agent-driven runs Clone the repository, select the 0.6.0 release tag, and run its verified build: ```bash theme={null} git clone https://github.com/markpollack/agent-bench.git cd agent-bench git checkout v0.6.0 ./mvnw clean verify ``` Benchmark setup, post-processing, and configured agent commands run as local host processes with your permissions. A workspace directory organizes files; it does not isolate them. For untrusted benchmark definitions or agents, supply your own disposable VM, CI runner, or other externally managed isolation. ## Pattern 1: End-to-End `run` Create an agent configuration: ```yaml theme={null} # agents/my-agent.yaml command: claude --print --dangerously-skip-permissions "Read INSTRUCTION.md and follow the instructions precisely." timeout: PT5M ``` Run the deterministic hello-world benchmark through the core CLI: ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="run --benchmark hello-world --agent agents/my-agent.yaml" ``` The lifecycle is `provide → setup → agent → post-processing → grade`. Results are written under `runs//`, including aggregate metadata and the preserved task workspace. For a jury containing `test-quality-llm`, use the agents-module entry point. Its `BenchApp` wires the real Agent Client-backed judge: ```bash theme={null} ./mvnw -q -pl agent-bench-agents exec:java \ -Dexec.args="run --benchmark code-coverage --agent agents/my-agent.yaml" ``` That path may invoke a model provider and requires the corresponding CLI credentials and runtime. ## Pattern 2: `provide`, External Agent, `grade` The split workflow lets another system own agent execution: ```bash theme={null} ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="provide --benchmark hello-world --task hello-world --workspace /tmp/hello-bench" cd /tmp/hello-bench your-agent "Read INSTRUCTION.md and complete the task" cd /path/to/agent-bench ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="grade --benchmark hello-world --task hello-world --workspace /tmp/hello-bench" ``` `provide` writes `INSTRUCTION.md` and `.bench-context.yaml`; the external agent changes the workspace; `grade` uses the benchmark's Agent Judge configuration. The split CLI path is intended for deterministic juries. LLM-backed judging is wired for agents-module `run` and `resume`. ## Pattern 3: Java API Applications can discover definitions and materialize the same Agent Judge jury directly: ```java theme={null} BenchmarkCatalog catalog = new BenchmarkCatalog(Path.of("benchmarks")); Benchmark benchmark = catalog.discover().stream() .filter(candidate -> candidate.name().equals("hello-world")) .findFirst() .orElseThrow(); JudgeFactory factory = new JudgeFactory(); Judge judge = factory.createFromConfig(benchmark.juryConfig()); Judgment judgment = judge.judge( JudgmentContext.builder().workspace(Path.of("/tmp/hello-bench")).build()); ``` `RunCommand`, `ProvideCommand`, `GradeCommand`, `CompareCommand`, and the benchmark/result records are also available for programmatic orchestration. ## Result Layout ```text theme={null} runs// result.json run-metadata.json bench.lock tasks/ / result.json workspace/ ``` Commands, timeouts, journals, and the trust boundary Commands and entry-point differences Agent Judge tiers, policies, and custom judges # Jury System Source: https://lab.pollack.ai/docs/agent-bench/jury-system How Agent Bench grades task workspaces with Agent Judge ## Task, Agent, Verifier Agent Bench follows a Terminal-Bench-inspired separation: 1. A task defines instructions, optional workspace material, setup/post-processing, and metadata. 2. An agent changes the prepared workspace. 3. A verifier grades the result. The verifier is an [Agent Judge](/projects/agent-judge) `Judge` or jury created by `JudgeFactory`. Agent Bench preserves Agent Judge's `Judgment`, jury, tier-policy, and voting semantics. The 0.6.0 release retains deterministic judges, simple and cascaded juries, and the agents module's Agent Client-backed LLM judge. ## Cascaded Tiers | Policy | Behavior | | -------------------- | ---------------------------------------------------- | | `REJECT_ON_ANY_FAIL` | Stop the cascade when any check fails | | `ACCEPT_ON_ALL_PASS` | Continue when all checks pass | | `FINAL_TIER` | Use the last tier's aggregated result as the verdict | ```yaml theme={null} jury: tiers: - name: build policy: REJECT_ON_ANY_FAIL checks: - type: maven-build goals: [clean, test] - name: test-quality policy: FINAL_TIER checks: - type: test-quality-llm prompt: prompts/judge-practice-adherence.txt model: claude-sonnet-4-6 ``` ## Registered Judge Types | Type | Implementation | Purpose | | ----------------------- | ------------------------------- | ------------------------------------------------------------------ | | `file-exists` | Agent Judge | Verify a workspace path exists | | `file-content` | Agent Judge | Compare file content exactly or by containment | | `maven-build` | Agent Judge `BuildSuccessJudge` | Run Maven goals and require success | | `coverage-preservation` | Agent Judge | Require coverage not to regress from the baseline | | `coverage-improvement` | Agent Judge | Require coverage to meet a threshold | | `test-quality-llm` | `agent-bench-agents` | Score practice adherence with a real Agent Client-backed LLM judge | The core CLI uses deterministic Agent Judge implementations and an abstaining LLM registration for end-to-end execution. The agents-module CLI registers `TestQualityJudge` for LLM-graded `run` and `resume` operations. ## Custom Judges Custom judges implement Agent Judge's `Judge` interface and can be registered without changing the benchmark format: ```java theme={null} public final class MyJudge implements Judge { @Override public Judgment judge(JudgmentContext context) { return Judgment.builder() .pass() .reasoning("Workspace satisfies the custom rule") .build(); } } JudgeFactory factory = new JudgeFactory(); factory.register("my-check", config -> new MyJudge()); ``` ## Benchmark and Task YAML ```yaml theme={null} schema: bench.benchmark.v1 name: my-benchmark version: "1.0" default-timeout: PT10M jury: checks: - type: file-exists path: expected.txt ``` ```yaml theme={null} schema: bench.task.v1 id: my-task difficulty: easy instruction: | Create expected.txt. timeout: PT10M metadata: key: value setup: - "command before the agent" post: - "command after the agent" ``` Setup, post-processing, configured agents, and process-based judges execute locally with the invoking user's permissions. A task workspace is not a security boundary; externally isolate untrusted definitions or agents. # What's New Source: https://lab.pollack.ai/docs/agent-bench/whats-new Release notes for Agent Bench ## 0.6.0 (2026-08-18) * Removes the unused `DockerSandbox` API and the direct Testcontainers/docker-java dependency. * Keeps local process execution, the `Sandbox` abstraction, `LocalSandbox`, Agent Judge grading, the agents module, and its real Agent Client-backed LLM judge. * Documents that workspaces are not security boundaries and recommends externally managed isolation for untrusted benchmark definitions or agents. * Aligns the dependency floors with Jackson 2.21.6, Jackson 3.1.6, Agent Judge 0.14.0, and Agent Client 0.26.0. * Publishes `agent-bench-core` and `agent-bench-agents` 0.6.0, plus the aggregate CycloneDX 1.6 SBOM. ## 0.5.0 (2026-08-17) * Added the Spring Boot upgrade benchmark candidate and retained its owner-operated external baseline-aware grader distinction. * Published the root aggregate CycloneDX SBOM and aligned Agent Judge and Agent Client dependencies. * Released two Maven modules: `agent-bench-core` and `agent-bench-agents`. ## 0.4.0 (2026-06-06) * Aligned Jackson 2 and Agent Client dependencies. ## 0.3.0 (2026-05-19) * Migrated to the `markpollack` organization, `io.github.markpollack` Maven coordinates, and the Business Source License 1.1 for new development. * Added Central metadata and documentation links. ## 0.2.1 (2026-03-31) * Initial release. # Defaults Philosophy Source: https://lab.pollack.ai/docs/agent-client/explanation/defaults-philosophy LOOSE vs STRICT mode — how AgentClient manages default permissiveness across providers ## The Problem Each agentic CLI has its own safety controls with different defaults. Without coordination, some providers work out of the box while others block on preconditions that users don't expect: | Provider | Autonomous Flag | Non-Git Directory | Out of Box? | | ----------- | --------------- | ----------------------------- | ----------- | | Claude Code | `yolo=true` | Works | Yes | | Codex | `fullAuto=true` | **Blocked** by `skipGitCheck` | **No** | | Gemini CLI | `yolo=true` | Works | Yes | A user switching from Claude to Codex hits a wall on the simplest task — "create a file" — with no obvious fix. ## AgentClientMode `AgentClientMode` is a portable enum that controls default permissiveness across all providers: ```java theme={null} public enum AgentClientMode { LOOSE, // Works out of the box in any directory STRICT // Requires explicit opt-in to risky operations } ``` **Default: `LOOSE`** — optimized for evaluation and development, where friction during onboarding is the primary failure mode. ### What Each Mode Does Permissive defaults that minimize preconditions: | Provider | Effect | | -------- | -------------------------------------------- | | Codex | `skipGitCheck=true` — works in any directory | | Claude | No change (already permissive) | | Gemini | No change (already permissive) | Conservative defaults for production environments: | Provider | Effect | | -------- | ---------------------------------------------- | | Codex | `skipGitCheck=false` — requires git repository | | Claude | No change currently | | Gemini | No change currently | ### Configuration Set the mode via Spring properties: ```yaml theme={null} agent-client: codex: mode: strict # or loose (default) ``` ## Precedence Rules Options are resolved in this order (first wins): 1. **Explicit goal options** — passed at call time via `AgentClient.run(goal, options)` 2. **Builder defaults** — set on `AgentClient.builder().defaultOptions()` 3. **Spring properties** — provider-specific values in `application.yml` 4. **Mode-derived defaults** — LOOSE or STRICT baseline 5. **Hardcoded defaults** — built into each provider's options class ### STRICT Is a Baseline, Not a Lock Explicit property overrides **always** take precedence over mode-derived defaults: ```yaml theme={null} agent-client: codex: mode: strict skip-git-check: true # This wins — overrides STRICT's default ``` In this configuration, Codex gets STRICT defaults for everything **except** the git check, which the user explicitly enabled. This is intentional — if you set a property explicitly, you made a conscious choice. If you set `mode: strict` expecting it to enforce all safety controls, audit your explicit property overrides. Any property set directly will pierce the mode. ### SDK Layer Stays Neutral The mode system operates at the **agent-models** layer, not the SDK layer. Provider SDKs (`codex-cli-sdk`, `claude-agent-sdk`, `gemini-cli-sdk`) always reflect their CLI's native defaults. Direct SDK consumers are never affected by `AgentClientMode`. This means: * `ExecuteOptions.builder().build()` → `skipGitCheck=false` (Codex CLI native default) * `CodexAgentProperties` with `mode=LOOSE` → `isSkipGitCheck()` returns `true` (mode-derived) ## Migration from Pre-0.14.0 Before 0.14.0, Codex defaulted to `skipGitCheck=false`. If you relied on this: ```yaml theme={null} # Restore pre-0.14.0 Codex behavior agent-client: codex: mode: strict ``` Or override the specific property: ```yaml theme={null} agent-client: codex: skip-git-check: false ``` # Getting Started Source: https://lab.pollack.ai/docs/agent-client/howto/getting-started Run your first agent task with Spring AI Agent Client in under 5 minutes ## Prerequisites * Java 17+ * Maven 3.9+ (or use the Maven wrapper) * A CLI agent installed and authenticated: * **Claude**: `claude auth login` ([install guide](https://docs.anthropic.com/en/docs/claude-code)) * **Codex**: `export OPENAI_API_KEY=sk-...` ([install guide](https://github.com/openai/codex)) * **Gemini**: `export GEMINI_API_KEY=...` ([install guide](https://github.com/google-gemini/gemini-cli)) ## 1. Add the Dependency Pick your provider. You need the agent model module for your provider: ```xml theme={null} io.github.markpollack agent-claude 0.29.0 ``` ```xml theme={null} io.github.markpollack agent-codex 0.29.0 ``` ```xml theme={null} io.github.markpollack agent-gemini 0.29.0 ``` ## 2. Create an Agent and Run a Task No Spring Boot required. Build a model, create a client, run a goal: ```java theme={null} import io.github.markpollack.agents.client.AgentClient; import io.github.markpollack.agents.client.AgentClientResponse; import io.github.markpollack.agents.claude.ClaudeAgentModel; import io.github.markpollack.agents.claude.ClaudeAgentOptions; public class HelloAgent { public static void main(String[] args) { ClaudeAgentOptions options = ClaudeAgentOptions.builder() .model("claude-sonnet-4-5") .yolo(true) .build(); ClaudeAgentModel model = ClaudeAgentModel.builder() .defaultOptions(options) .build(); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Agent Client!'" ); System.out.println("Success: " + response.isSuccessful()); } } ``` ```java theme={null} import io.github.markpollack.agents.client.AgentClient; import io.github.markpollack.agents.client.AgentClientResponse; import io.github.markpollack.agents.codex.CodexAgentModel; import io.github.markpollack.agents.codex.CodexAgentOptions; import io.github.markpollack.agents.codexsdk.CodexClient; public class HelloAgent { public static void main(String[] args) { CodexAgentOptions options = CodexAgentOptions.builder() .model("gpt-5-codex") .skipGitCheck(true) .build(); CodexAgentModel model = new CodexAgentModel( CodexClient.create(), options, null ); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Agent Client!'" ); System.out.println("Success: " + response.isSuccessful()); } } ``` ```java theme={null} import io.github.markpollack.agents.client.AgentClient; import io.github.markpollack.agents.client.AgentClientResponse; import io.github.markpollack.agents.gemini.GeminiAgentModel; import io.github.markpollack.agents.gemini.GeminiAgentOptions; import io.github.markpollack.agents.geminisdk.GeminiClient; public class HelloAgent { public static void main(String[] args) { GeminiAgentOptions options = GeminiAgentOptions.builder() .model("gemini-2.5-flash") .yolo(true) .build(); GeminiAgentModel model = new GeminiAgentModel( GeminiClient.create(), options, null ); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Agent Client!'" ); System.out.println("Success: " + response.isSuccessful()); } } ``` ## 3. Run It ```bash theme={null} mvn compile exec:java -Dexec.mainClass="HelloAgent" ``` The agent will create `hello.txt` in your working directory. ## With Spring Boot If you're building a Spring Boot application, use the starter dependencies instead. They auto-configure the `AgentApi` and provide an injectable `AgentClient.Builder`: ```xml theme={null} io.github.markpollack agent-starter-claude 0.29.0 ``` ```java theme={null} @Component public class MyAgentRunner implements CommandLineRunner { private final AgentClient.Builder agentClientBuilder; public MyAgentRunner(AgentClient.Builder agentClientBuilder) { this.agentClientBuilder = agentClientBuilder; } @Override public void run(String... args) { AgentClient client = agentClientBuilder.build(); AgentClientResponse response = client.run("Create hello.txt"); System.out.println("Success: " + response.isSuccessful()); } } ``` Configuration goes in `application.yml`: ```yaml theme={null} agent-client: claude: model: claude-sonnet-4-5 yolo: true ``` ## Next Steps * [Switching Providers](/docs/agent-client/howto/switching-providers) — Run the same task with different providers * [Structured Output](/docs/agent-client/howto/structured-output) — Get structured JSON responses from agents * [Configuration Reference](/docs/agent-client/reference/portable-options) — All available configuration options * [Tutorial: First Task](/docs/agent-client/tutorial/01-first-task) — Step-by-step walkthrough with verification # Structured Output Source: https://lab.pollack.ai/docs/agent-client/howto/structured-output How to get structured JSON responses from agent tasks using JSON Schema ## The Problem You want the agent to return structured data (not free-form text) so your application can parse and act on the result programmatically. ## Solution: JSON Schema Pass a JSON schema to the agent via Spring properties or the fluent API. The agent constrains its output to match the schema. ### Via Spring Properties ```yaml theme={null} agent-client: claude: json-schema: type: object properties: summary: type: string description: "A brief summary of what was done" files_changed: type: array items: type: string description: "List of files that were created or modified" success: type: boolean required: - summary - success ``` ### Via the Fluent API ```java theme={null} AgentClient agentClient = agentClientBuilder.build(); AgentClientResponse response = agentClient.goal("Analyze the project structure") .workingDirectory(projectDir) .run(); // Parse the structured response String result = response.getResult(); JsonNode json = objectMapper.readTree(result); String summary = json.get("summary").asText(); ``` **Provider support varies.** JSON schema structured output is currently supported by the **Claude** provider. Codex and Gemini may return free-form text even when a schema is configured. Check the provider-specific reference pages for current support. ## Provider-Specific Details | Provider | Structured Output | Configuration | | -------- | -------------------------------------- | ----------------------------------------------------------------------------------- | | Claude | JSON Schema via `json-schema` property | [Claude Reference](/docs/agent-client/reference/claude-reference#structured-output) | | Codex | Not currently supported | — | | Gemini | Not currently supported | — | # Switching Providers Source: https://lab.pollack.ai/docs/agent-client/howto/switching-providers How to run the same agent task with Claude, Codex, or Gemini using Maven profiles ## The Problem You want to run the same agent task with different providers without changing your Java code. ## Solution: Maven Profiles Define one Maven profile per provider. Each profile puts a different starter on the classpath. Your Java code uses only the portable `AgentClient` API. ### 1. Define Profiles in pom.xml ```xml theme={null} claude true io.github.markpollack agent-starter-claude ${agent-client.version} codex io.github.markpollack agent-starter-codex ${agent-client.version} gemini io.github.markpollack agent-starter-gemini ${agent-client.version} ``` ### 2. Add Per-Provider Configuration Create profile-specific YAML files alongside your main `application.yml`: ```yaml theme={null} agent-client: claude: model: claude-sonnet-4-5 yolo: true ``` ```yaml theme={null} agent-client: codex: model: gpt-5-codex full-auto: true ``` ```yaml theme={null} agent-client: gemini: model: gemini-2.5-flash yolo: true ``` Activate the matching Spring profile in your main `application.yml`: ```yaml theme={null} spring: profiles: active: claude # Matches the Maven profile ``` ### 3. Run with a Specific Provider ```bash theme={null} # Claude (default profile) ./mvnw spring-boot:run # Codex ./mvnw spring-boot:run -Pcodex -P'!claude' \ -Dspring-boot.run.arguments="--spring.profiles.active=codex" # Gemini ./mvnw spring-boot:run -Pgemini -P'!claude' \ -Dspring-boot.run.arguments="--spring.profiles.active=gemini" ``` Use `-P'!claude'` to deactivate the default Claude profile when switching to another provider. Only one provider starter should be on the classpath. ## Your Java Code Stays the Same The key design principle: **your Java code never imports provider-specific classes**. It only uses: * `AgentClient` / `AgentClient.Builder` * `AgentClientResponse` * `AgentOptions` (for portable options) Provider selection happens entirely through Maven profiles and Spring configuration. ## Working Example See [samples/create-file-multi-provider](https://github.com/markpollack/agent-client/tree/main/samples/create-file-multi-provider) for a complete working example, or the [agent-client-tutorial](https://github.com/markpollack/agent-client-tutorial) for progressive examples. # Antigravity Reference Source: https://lab.pollack.ai/docs/agent-client/reference/antigravity-reference Complete configuration reference for the Google Antigravity agent provider ## Overview The Antigravity agent wraps the [Google Antigravity CLI](https://antigravity.google) via `AntigravityAgentModel`. The binary is `agy`. Configure it through Spring properties under `agent-client.antigravity.*`. ```yaml theme={null} agent-client: antigravity: model: gemini-3.1-pro-high timeout: PT15M ``` ## Configuration Properties Prefix: `agent-client.antigravity` | Property | Type | Default | Description | | ------------------------------ | ----------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `mode` | `AgentClientMode` | — | Controls default permissiveness. Inherits from `agent-client.mode` if not set (default: `LOOSE`). | | `model` | `String` | `gemini-3.1-pro-high` | Model slug (`--model`). List them with `agy models`. | | `effort` | `String` | — | Reasoning effort (`--effort`): `low`, `medium`, `high`. **Ignored when the model slug already encodes it** — see below. | | `timeout` | `Duration` | `15m` | Timeout. Also drives `--print-timeout`. | | `dangerously-skip-permissions` | `Boolean` | — | `--dangerously-skip-permissions`. When not explicitly set, derived from mode. | | `execution-mode` | `ExecutionMode` | — | `--mode`: `accept-edits` or `plan` | | `sandbox` | `boolean` | `false` | `--sandbox` — terminal restrictions | | `executable-path` | `String` | — | Path to `agy` (auto-discovered if not set) | ## Effort is encoded in the model slug Antigravity names effort twice: in `--effort` and in slugs like `gemini-3.1-pro-high`. Passing both is a **hard error**: ``` invalid model selection (--model "gemini-3.1-pro-high" --effort "low"): --model gemini-3.1-pro-high conflicts with --effort=low ``` The run fails immediately with `"status": "ERROR"` and no output. So when the slug ends in `-low`, `-medium` or `-high`, the provider drops the flag and the slug wins. Portable `getEffort()` therefore behaves here exactly as it does for every other provider, and a caller does not need to know that this one CLI names effort twice. ## dangerouslySkipPermissions and Mode Interaction | `dangerously-skip-permissions` | `mode` | Effective Value | | ------------------------------ | -------- | --------------- | | explicit | any | as set | | unset | `STRICT` | `false` | | unset | `LOOSE` | `true` | Declining to auto-approve does not do what the flag name suggests. Antigravity's headless default is not to stop on an unapprovable tool call — it is to **refuse that call and carry on**. A `STRICT` unattended run is therefore not safer, it is quietly partial. It remains the default only because a caller that has not thought about permissions should not be handed unrestricted execution, and because the provider reports the refusals rather than swallowing them. ## Detecting a refused run The provider scans both the envelope's `error` field and stderr for refusal notices, and surfaces the result: | Field | Notes | | ------------------- | ---------------------------------------------- | | `softDenied` | `true` when at least one tool call was refused | | `permissionNotices` | The refusal messages, verbatim | The documentation says these notices go to stderr. agy 1.1.13 puts some of them in the envelope's `error` field with stderr empty, and others on stderr with `status: CANCELED`. Both channels are read. ## The status field is not the verdict In agy 1.1.13, `status` behaves like a sticky execution-error indicator, not a verdict on the completed task. A tool call that fails can leave the terminal envelope at `"status":"ERROR"` even after later tool calls complete and the agent produces a valid review. Conversely, a refused tool call has been observed with `"status":"SUCCESS"` and no response. Trusting the field discards good recovered work; ignoring it hides real failures. `isSuccessful()` therefore requires all of the following: * the process exited with code 0 * the run produced a non-empty response * no tool call was refused * the stream contains no tool error left unrecovered by a later completed tool call An `ERROR` envelope without stream evidence is treated conservatively as unrecovered. The CLI's own account remains available as `status`, `reportedSuccessful` and `error`; `softDenied` and `unrecoveredError` expose the two independent reasons the derived result can reject it. ## Portable Option Mapping | Portable option | Antigravity flag | | ------------------------- | ------------------------------------------------------------- | | `getModel()` | `--model` | | `getEffort()` | `--effort`, **dropped when the model slug encodes effort** | | `getJsonSchema()` | `--json-schema` (serialized inline) | | `isAutoApprove()` | `--dangerously-skip-permissions` | | `getTimeout()` | process timeout **and** `--print-timeout` | | `getWorkingDirectory()` | `--add-dir` **and** the process working directory — see below | | `getSystemInstructions()` | prepended to the goal (no system-prompt flag exists) | | `getMaxTurns()` | not supported | `--print-timeout` defaults to five minutes, well short of a real task, so it is always set from the caller's timeout. Left alone, a truncated run would be indistinguishable from the agent finishing early. ## The working directory must be declared, not just set There is no `--cwd`, but setting the process working directory is **not sufficient**. With no active workspace, `agy` diverts every write to a shared scratch directory under `~/.gemini/antigravity-cli/` and still reports success. The CLI says so plainly in its own output — *"I placed it in your scratch directory … since there wasn't an active workspace"* — and artifacts from earlier runs accumulate there, so work leaks between runs. Reads resolve against the process working directory, which is why a narrow check looks fine while writes are going somewhere else entirely. The provider therefore declares the working directory with `--add-dir` as well as setting the process cwd. Both are required. This was found by running the provider parity TCK, not by reading the CLI's documentation. A run that writes to the wrong place and reports success is the failure mode this provider is most exposed to — see also soft denial below. ## Result Metadata `getProviderFields()` carries `inputTokens`, `outputTokens`, `thinkingTokens`, `cacheReadTokens`, `totalTokens`, `numTurns`, `status`, `reportedSuccessful`, `error`, `softDenied`, `unrecoveredError`, `permissionNotices`, `exitCode` and `structured`. `getSessionId()` carries the conversation id, which `AntigravityClient.resume(conversationId, ...)` accepts via `--conversation`. ## Availability Since **0.28.0**. ## Installation Install from [antigravity.google](https://antigravity.google) — the npm distribution is discontinued — and authenticate once interactively. Discovery checks `ANTIGRAVITY_CLI_PATH`, then `which agy`, then `~/.local/bin/agy`. Support files live under `~/.gemini/antigravity-cli`, shared with the Gemini CLI, so that directory existing proves nothing about `agy` being installed. # Claude Reference Source: https://lab.pollack.ai/docs/agent-client/reference/claude-reference Complete configuration reference for the Claude Code agent provider ## Overview The Claude agent wraps the [Claude Code CLI](https://docs.anthropic.com/en/docs/claude-code) via `ClaudeAgentModel`. Configure it through Spring properties under `agent-client.claude.*`. ```yaml theme={null} agent-client: claude: model: claude-sonnet-4-5 timeout: PT5M yolo: true ``` ## Configuration Properties Prefix: `agent-client.claude` | Property | Type | Default | Description | | ---------------------- | -------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | `model` | `String` | `claude-sonnet-4-5` | Claude model to use for agent tasks | | `timeout` | `Duration` | `5m` | Timeout for agent task execution | | `yolo` | `boolean` | `true` | Bypass all permission checks | | `executable-path` | `String` | — | Path to the Claude CLI executable (auto-discovered if not set) | | `effort` | `String` | — | Reasoning effort level (`--effort`): `low`, `medium`, `high`, `xhigh`, `max`. Overrides the portable `effort` when both are set | | `max-thinking-tokens` | `Integer` | — | Maximum thinking tokens for extended thinking mode | | `system-prompt` | `String` | — | System prompt to use for the agent | | `allowed-tools` | `List` | `[]` | Tools that are allowed to be used | | `disallowed-tools` | `List` | `[]` | Tools that are not allowed to be used | | `permission-mode` | `String` | — | Permission mode for tool execution (overrides `yolo` if set) | | `json-schema` | `Map` | — | JSON schema for structured output | | `max-tokens` | `Integer` | — | Maximum tokens for the response | | `max-turns` | `Integer` | — | Maximum number of agentic turns before stopping | | `max-budget-usd` | `Double` | — | Maximum budget in USD before stopping | | `fallback-model` | `String` | — | Fallback model if the primary model is unavailable | | `append-system-prompt` | `String` | — | Additional text appended to the default system prompt | ### Advanced Options These options provide full parity with the Claude Code Python SDK: | Property | Type | Default | Description | | ----------------------------- | -------------------- | ------- | ----------------------------------------------------------------- | | `add-dirs` | `List` | `[]` | Additional directories to include in Claude's context | | `settings` | `String` | — | Custom settings file path | | `permission-prompt-tool-name` | `String` | — | Permission prompt tool name for interactive permission handling | | `extra-args` | `Map` | — | Arbitrary extra CLI arguments (keys are flag names without `--`) | | `env` | `Map` | — | Custom environment variables for the CLI process | | `max-buffer-size` | `Integer` | — | Maximum buffer size for JSON parsing in bytes (default 1MB) | | `user` | `String` | — | Unix user to run the CLI process as (requires sudo configuration) | | `trace-dir` | `String` | — | Directory for JSONL trace files. Each invocation writes one file. | ## Permission Modes The `permission-mode` property overrides `yolo` when set. Available modes: | Mode | Description | | ------------------- | ------------------------------------------------------ | | `bypassPermissions` | Skip all permission checks (equivalent to `yolo=true`) | | `default` | Use Claude Code's default permission behavior | ```yaml theme={null} agent-client: claude: permission-mode: bypassPermissions # yolo is ignored when permission-mode is set ``` ## Tool Filtering Control which tools Claude can use: ```yaml theme={null} agent-client: claude: allowed-tools: - Read - Write - Bash disallowed-tools: - WebSearch ``` `allowed-tools` and `disallowed-tools` are mutually exclusive in practice. If both are set, `allowed-tools` takes precedence. ## Budget Controls Limit agent execution costs and turns: ```yaml theme={null} agent-client: claude: max-turns: 10 # Stop after 10 agentic turns max-budget-usd: 0.50 # Stop after $0.50 spent max-tokens: 4096 # Limit response tokens ``` ## Structured Output Request structured JSON output using a JSON schema: ```yaml theme={null} agent-client: claude: json-schema: type: object properties: summary: type: string files_changed: type: array items: type: string ``` ## Trace Files Enable JSONL trace output to capture tool calls, thinking blocks, and result metrics per invocation: ```yaml theme={null} agent-client: claude: trace-dir: /path/to/traces ``` Each `call()` writes one file like `agent-run-20260528-143000-123-a1b2c3d4.jsonl` containing Claude-specific events (`tool_use`, `tool_result`, `text`, `thinking`, `result`). The file path is available in the response metadata: ```java theme={null} String tracePath = (String) response.getMetadata().getProviderFields().get("tracePath"); ``` Or via the builder API: ```java theme={null} ClaudeAgentModel model = ClaudeAgentModel.builder() .traceDir(Path.of("traces")) .build(); ``` Trace files are flushed per event during parsing, so you can `tail -f` them during long-running sessions. When used with [Agent Workflow](/docs/agent-workflow/trace-capture), trace paths propagate automatically through the workflow journal. These are Claude-specific JSONL traces, not provider-neutral journal events. See [Agent Journal](/projects/agent-journal) for the portable event layer. ## Authentication ```bash theme={null} claude auth login ``` The Claude CLI uses its own session token. No environment variables needed. ```bash theme={null} export ANTHROPIC_API_KEY=sk-ant-... ``` Do not mix session auth and API key auth — this can cause authentication conflicts. # Codex Reference Source: https://lab.pollack.ai/docs/agent-client/reference/codex-reference Complete configuration reference for the OpenAI Codex agent provider ## Overview The Codex agent wraps the [OpenAI Codex CLI](https://github.com/openai/codex) via `CodexAgentApi`. Configure it through Spring properties under `agent-client.codex.*`. ```yaml theme={null} agent-client: codex: model: gpt-5-codex timeout: PT5M full-auto: true ``` ## Configuration Properties Prefix: `agent-client.codex` | Property | Type | Default | Description | | ------------------ | ----------------- | ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `mode` | `AgentClientMode` | — | Controls default permissiveness. Inherits from `agent-client.mode` if not set (default: `LOOSE`). | | `model` | `String` | `gpt-5-codex` | Model to use for Codex execution | | `timeout` | `Duration` | `5m` | Timeout for agent task execution | | `full-auto` | `boolean` | `true` | Enable full-auto mode (workspace-write sandbox + never approval) | | `skip-git-check` | `Boolean` | — | Skip git repository check. When not explicitly set, derived from mode (see below). | | `reasoning-effort` | `String` | — | Model reasoning effort (`model_reasoning_effort` config override): `minimal`, `low`, `medium`, `high`, `xhigh`. Overrides the portable `effort` when both are set | | `executable-path` | `String` | — | Path to the Codex CLI executable (auto-discovered if not set) | ## skipGitCheck and Mode Interaction The `skip-git-check` property has special behavior — its effective value depends on `AgentClientMode` when not explicitly set: | `skip-git-check` | `mode` | Effective Value | Behavior | | ------------------ | ----------------- | --------------- | ----------------------- | | `true` (explicit) | any | `true` | Works in any directory | | `false` (explicit) | any | `false` | Requires git repository | | not set | `LOOSE` (default) | `true` | Works in any directory | | not set | `STRICT` | `false` | Requires git repository | **Explicit always wins.** Setting `skip-git-check` directly always overrides the mode-derived value. See [Defaults Philosophy](/docs/agent-client/explanation/defaults-philosophy) for the precedence rationale. ### Why This Matters Codex is the only provider that blocks execution in non-git directories by default. Without `skipGitCheck=true`, attempting to create a file in a temporary directory fails: ``` Error: Not a git repository (or any of the parent directories) ``` The `LOOSE` mode (default) automatically resolves this, so users can start with Codex without hitting this wall. **Migration note (pre-0.14.0):** Before the mode system, Codex required `skip-git-check: true` explicitly for non-git directories. With `LOOSE` mode (now the default), this is automatic. To restore the old behavior, set `mode: strict` or `skip-git-check: false`. ## Full-Auto Mode When `full-auto=true` (default), Codex runs with: * **Workspace-write sandbox** — can write files in the working directory * **Never asks for approval** — autonomous execution ```yaml theme={null} agent-client: codex: full-auto: true # Default — autonomous execution ``` The Codex CLI `exec` subcommand does **not** support `--ask-for-approval`. Only `--full-auto` and `--sandbox` control execution behavior. ## Authentication ```bash theme={null} export OPENAI_API_KEY=sk-... ``` The Codex CLI requires an OpenAI API key via the `OPENAI_API_KEY` environment variable. # Gemini Reference Source: https://lab.pollack.ai/docs/agent-client/reference/gemini-reference Complete configuration reference for the Google Gemini agent provider ## Overview The Gemini agent wraps the [Gemini CLI](https://github.com/google-gemini/gemini-cli) via `GeminiAgentApi`. Configure it through Spring properties under `agent-client.gemini.*`. ```yaml theme={null} agent-client: gemini: model: gemini-2.5-flash timeout: PT5M yolo: true ``` ## Configuration Properties Prefix: `agent-client.gemini` | Property | Type | Default | Description | | ----------------- | ---------- | ------------------ | -------------------------------------------------------------- | | `model` | `String` | `gemini-2.5-flash` | Gemini model to use for agent tasks | | `timeout` | `Duration` | `5m` | Timeout for agent task execution | | `yolo` | `boolean` | `true` | Bypass all permission checks | | `executable-path` | `String` | — | Path to the Gemini CLI executable (auto-discovered if not set) | | `temperature` | `Double` | — | Temperature for controlling response randomness (0.0–1.0) | | `max-tokens` | `Integer` | — | Maximum number of tokens to generate in the response | ## Yolo Mode When `yolo=true` (default), Gemini runs autonomously without permission prompts: ```yaml theme={null} agent-client: gemini: yolo: true # Default — no permission prompts ``` Set `yolo: false` if you want Gemini to ask for confirmation before executing commands. ## Temperature Control response randomness with the `temperature` property: ```yaml theme={null} agent-client: gemini: temperature: 0.2 # More deterministic ``` | Value | Behavior | | ----- | --------------------------------------- | | `0.0` | Most deterministic — consistent outputs | | `0.5` | Balanced | | `1.0` | Most creative — varied outputs | Temperature support depends on the Gemini CLI version. Check your CLI version with `gemini --version`. ## Mode System Gemini works in any directory with default settings — no mode-derived behavior currently exists. The `LOOSE` and `STRICT` modes are a no-op for Gemini today. Future `STRICT` mode knobs for Gemini will be added when there is a concrete safety control to gate (following the [promotion rubric](/docs/agent-client/reference/portable-options#promotion-rubric)). ## Authentication ```bash theme={null} export GEMINI_API_KEY=... # or export GOOGLE_API_KEY=... ``` The Gemini CLI accepts either `GEMINI_API_KEY` or `GOOGLE_API_KEY` environment variables. # Grok Reference Source: https://lab.pollack.ai/docs/agent-client/reference/grok-reference Complete configuration reference for the xAI Grok agent provider ## Overview The Grok agent wraps the [xAI Grok CLI](https://grok.com/cli) via `GrokAgentModel`. Configure it through Spring properties under `agent-client.grok.*`. ```yaml theme={null} agent-client: grok: model: grok-4.6 reasoning-effort: high timeout: PT10M ``` ## Configuration Properties Prefix: `agent-client.grok` | Property | Type | Default | Description | | -------------------- | ----------------- | ---------- | ----------------------------------------------------------------------------------------------------------------- | | `mode` | `AgentClientMode` | — | Controls default permissiveness. Inherits from `agent-client.mode` if not set (default: `LOOSE`). | | `model` | `String` | `grok-4.6` | Model to use (`--model`) | | `reasoning-effort` | `String` | — | Reasoning effort (`--reasoning-effort`): `low`, `medium`, `high`. The portable `effort` values map here directly. | | `timeout` | `Duration` | `5m` | Timeout for agent task execution | | `permission-mode` | `PermissionMode` | — | `--permission-mode`. When not explicitly set, derived from mode (see below). | | `max-turns` | `Integer` | — | Maximum agent turns (`--max-turns`) | | `disable-web-search` | `boolean` | `false` | Disable web search and fetch tools (`--disable-web-search`) | | `executable-path` | `String` | — | Path to the Grok CLI executable (auto-discovered if not set) | ## permissionMode and Mode Interaction | `permission-mode` | `mode` | Effective Value | Behavior | | ----------------- | -------- | ------------------- | ------------------------------------------------------------------ | | explicit | any | as set | Always wins | | unset | `STRICT` | `default` | The CLI's own default — a tool call needing approval stops the run | | unset | `LOOSE` | `bypassPermissions` | Every tool call is approved without prompting | The portable `isAutoApprove()` maps to `bypassPermissions`, the only mode that grants every tool call without a prompt. ## Portable Option Mapping | Portable option | Grok flag | | ------------------------- | ------------------------------------------------------------------------------- | | `getModel()` | `--model` | | `getEffort()` | `--reasoning-effort` (direct passthrough — `low`/`medium`/`high` are all valid) | | `getMaxTurns()` | `--max-turns` | | `getJsonSchema()` | `--json-schema` (serialized inline; no temp file) | | `isAutoApprove()` | `--permission-mode bypassPermissions` | | `getTimeout()` | process timeout | | `getWorkingDirectory()` | `--cwd` and the process working directory | | `getSystemInstructions()` | prepended to the goal — see below | ### System instructions are prepended, not passed as a flag Grok has `--system-prompt-override`, but it *replaces* the CLI's own system prompt rather than adding to it, which strips the agent's tool instructions. Portable `systemInstructions` mean "also tell it this", so they are prepended to the goal — the same choice every other provider here makes. ## Result Metadata Grok returns a JSON envelope in headless mode, so nothing is scraped from log text. `AgentResponseMetadata.getProviderFields()` carries: | Field | Notes | | ------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- | | `costUsd` | **A real per-run cost.** Grok is the only CLI in this family that reports one, so callers do not need a price table. | | `inputTokens`, `outputTokens`, `thinkingTokens` | From the envelope's `usage` block | | `cacheReadTokens`, `cacheCreationTokens`, `totalTokens` | | | `numTurns`, `stopReason`, `exitCode`, `successful` | | | `structured` | `false` when the output was not the expected envelope; the raw text is preserved as the response | `getModel()` reports the model that actually ran (from the envelope's `modelUsage` key), which may differ from the requested slug when the CLI expands an alias. `getSessionId()` carries the session UUID. ## Sessions Grok accepts a caller-supplied session UUID (`--session-id`) for a new conversation and resumes with `--resume `, so the id never has to be recovered from output. `GrokClient.resume(sessionId, prompt, options)` exposes this. Grok does not currently implement `AgentSessionRegistry`. `grok models` reports "You are not authenticated" even on an authenticated install. The provider's health check therefore uses `grok --version`; using `models` would reject a working CLI. ## Availability Since **0.28.0**. ## Installation Install the CLI from [grok.com/cli](https://grok.com/cli) and authenticate once with `grok login`. The provider discovers the executable via `GROK_CLI_PATH`, then `which grok`, then `~/.local/bin/grok` and `~/.grok/bin/grok`. # Portable Options Reference Source: https://lab.pollack.ai/docs/agent-client/reference/portable-options AgentOptions interface, mode system, and the precedence rules that govern agent behavior across providers ## AgentOptions `AgentOptions` is the portable interface that all provider-specific options extend. It defines the common surface that works identically across Claude, Codex, and Gemini. ```java theme={null} public interface AgentOptions { String getModel(); Duration getTimeout(); } ``` Each provider adds its own options beyond this interface — see the provider-specific reference pages for details. ## Configuration Precedence Agent behavior is determined by a layered precedence system. Higher layers override lower ones: ``` 1. Explicit goal options (highest — per-request) 2. Builder configuration (per-client instance) 3. Spring properties (application.yml / env vars) 4. Mode-derived defaults (from AgentClientMode) 5. Hardcoded defaults (lowest — in code) ``` **Explicit always wins.** If you set `agent-client.codex.skip-git-check=true` alongside `agent-client.mode=strict`, the explicit property wins. See [Defaults Philosophy](/docs/agent-client/explanation/defaults-philosophy) for the full rationale. ### Example: Precedence in Action ```yaml theme={null} agent-client: mode: strict codex: skip-git-check: true # Explicit property overrides STRICT model: o3-mini # Overrides hardcoded default (gpt-5-codex) ``` Result: Codex runs in STRICT mode **except** `skipGitCheck` is `true` because the explicit property takes precedence. ## Mode System `AgentClientMode` is a portable enum controlling default permissiveness: | Mode | Behavior | When to Use | | ----------------- | ----------------------------------------------- | ----------------------------------- | | `LOOSE` (default) | Minimize preconditions — works in any directory | Evaluation, development, tutorials | | `STRICT` | Conservative — requires explicit opt-in | Production, CI, shared environments | ```yaml theme={null} agent-client: mode: loose # or strict ``` See [Defaults Philosophy](/docs/agent-client/explanation/defaults-philosophy) for detailed mode-vs-property interaction examples. ## Promotion Rubric Provider-specific options graduate to the portable `AgentOptions` interface when **all three** conditions hold: 1. **Two or more providers** have a semantic equivalent 2. **Absence causes failures** on easy-tier benchmarks (evidence, not speculation) 3. The option **can be expressed** without leaking provider-specific concepts Options that don't meet the rubric remain in the provider-specific namespace indefinitely. This is intentional — not every option should be portable. ### Current Portable Options | Option | Claude | Codex | Gemini | Status | | --------- | ------------------- | ------------------------ | ------------------------ | --------------------------------------------------------- | | `model` | `claude-sonnet-4-5` | `gpt-5-codex` | `gemini-2.5-flash` | Portable | | `timeout` | 5m | 5m | 5m | Portable | | `effort` | `--effort` | `model_reasoning_effort` | ignored (no effort knob) | Portable (`low`/`medium`/`high`; native ranges are wider) | ### Provider-Specific (Not Promoted) | Option | Provider | Why Not Portable | | ------------------- | -------------- | ------------------------------------------- | | `skipGitCheck` | Codex | Only Codex has a git directory gate | | `maxThinkingTokens` | Claude | Claude-specific extended thinking | | `yolo` | Claude, Gemini | Codex uses `fullAuto` — different semantics | | `temperature` | Gemini | Not exposed by Claude/Codex CLIs | ## Provider Reference Pages 18 properties — `agent-client.claude.*` 6 properties — `agent-client.codex.*` 6 properties — `agent-client.gemini.*` # Agent Sessions Source: https://lab.pollack.ai/docs/agent-client/reference/sessions Persistent, multi-turn agent conversations with session lifecycle management ## Overview Agent Sessions enable **persistent, multi-turn conversations** with CLI agents. Think of them like `HttpSession` — the session maintains state across prompts so you can have iterative dialogues with an agent instead of fire-and-forget single tasks. Without sessions, each `AgentClient.goal().run()` call starts a fresh conversation. With sessions, you can: * Send follow-up prompts that build on previous context * Resume conversations after transport failures * Manage session lifecycle with automatic stale cleanup **Available since version 0.10.0.** Currently implemented for the **Claude** agent provider. Other providers will follow. ## Core Interfaces ### AgentSession A single persistent conversation. Created via `AgentSessionRegistry.create()` — never instantiated directly. ```java theme={null} public interface AgentSession extends AutoCloseable { String getSessionId(); Path getWorkingDirectory(); AgentSessionStatus getStatus(); AgentResponse prompt(String message); AgentSession resume(); AgentSession fork(); void close(); } ``` | Method | Description | | ----------------------- | ---------------------------------------------------------- | | `getSessionId()` | Unique session ID, assigned eagerly at creation | | `getWorkingDirectory()` | Immutable working directory the session operates in | | `getStatus()` | Current lifecycle status (`ACTIVE`, `DEAD`, or `RESUMED`) | | `prompt(message)` | Send a follow-up in the same conversation context | | `resume()` | Resurrect a `DEAD` session — restores conversation history | | `fork()` | Branch the conversation (not yet implemented) | | `close()` | Close the session and release resources | ### AgentSessionRegistry Factory and lifecycle manager for sessions. Analogous to `SessionRepository` in Spring Session. ```java theme={null} public interface AgentSessionRegistry { AgentSession create(Path workingDirectory); Optional find(String sessionId); void evict(String sessionId); void evictStale(Duration inactiveSince); } ``` | Method | Description | | ---------------------- | --------------------------------------------------------------------- | | `create(path)` | Start a new session — eagerly connects to CLI and captures session ID | | `find(id)` | Look up an existing session | | `evict(id)` | Remove and close a session | | `evictStale(duration)` | Evict sessions inactive longer than the threshold | ### AgentSessionStatus ``` ACTIVE → Session is connected and ready for prompts DEAD → Transport died; call resume() to resurrect RESUMED → Was dead, now active again after resume() ``` ## Usage ### Create a Registry and Session ```java theme={null} import io.github.markpollack.agents.claude.ClaudeAgentSessionRegistry; import io.github.markpollack.agents.model.AgentSession; import io.github.markpollack.agents.model.AgentResponse; // Build a registry (configure once, create many sessions) ClaudeAgentSessionRegistry registry = ClaudeAgentSessionRegistry.builder() .timeout(Duration.ofMinutes(5)) .build(); // Create a session — eagerly establishes CLI connection AgentSession session = registry.create(Path.of("/my/project")); System.out.println("Session ID: " + session.getSessionId()); ``` ### Multi-Turn Conversation ```java theme={null} // First prompt AgentResponse r1 = session.prompt("Create a Spring Boot REST controller for /api/users"); // Follow-up — Claude remembers the previous context AgentResponse r2 = session.prompt("Add input validation with @Valid"); // Another follow-up AgentResponse r3 = session.prompt("Now write tests for the controller"); // Clean up session.close(); registry.evict(session.getSessionId()); ``` Each `prompt()` call continues the same conversation — the agent sees the full history of what it built in earlier turns. ### Resuming a Dead Session If the CLI process dies (crash, timeout, network issue), the session transitions to `DEAD`. You can resurrect it: ```java theme={null} AgentSession session = registry.create(Path.of("/my/project")); String savedId = session.getSessionId(); try { session.prompt("Refactor the service layer"); } catch (IllegalStateException e) { // Transport died — session is now DEAD if (session.getStatus() == AgentSessionStatus.DEAD) { session.resume(); // Spawns fresh process, restores history // Status is now RESUMED — ready for prompts again session.prompt("Continue the refactoring"); } } ``` ### Finding an Existing Session ```java theme={null} // Later in the application lifecycle Optional found = registry.find(savedId); found.ifPresent(s -> { AgentResponse response = s.prompt("What files did you modify?"); System.out.println(response.getText()); }); ``` ## Session Lifecycle ``` create() │ ▼ ┌────────┐ │ ACTIVE │◄─────────────┐ └───┬────┘ │ │ │ prompt() works resume() transport dies │ │ │ ▼ │ ┌────────┐ ┌────────┐ │ DEAD │────────►│RESUMED │ └───┬────┘ └────────┘ │ close() / evict() │ ▼ [removed] ``` ## Spring Integration ### Bean Configuration ```java theme={null} @Configuration @EnableScheduling public class AgentSessionConfig { @Bean public ClaudeAgentSessionRegistry agentSessionRegistry() { return ClaudeAgentSessionRegistry.builder() .timeout(Duration.ofMinutes(10)) .build(); } } ``` ### Stale Session Cleanup Use `@Scheduled` to periodically evict inactive sessions and prevent resource leaks: ```java theme={null} @Component public class SessionCleanup { private final ClaudeAgentSessionRegistry registry; public SessionCleanup(ClaudeAgentSessionRegistry registry) { this.registry = registry; } @Scheduled(fixedRate = 300_000) // Every 5 minutes public void cleanupStaleSessions() { registry.evictStale(Duration.ofMinutes(30)); } } ``` ### Startup Health Probe Use a disposable session to verify CLI availability at startup: ```java theme={null} @Component public class AgentHealthCheck { private final ClaudeAgentSessionRegistry registry; public AgentHealthCheck(ClaudeAgentSessionRegistry registry) { this.registry = registry; } @EventListener(ApplicationReadyEvent.class) public void verifyCliAvailable() { try { AgentSession probe = registry.create(Path.of("/tmp")); probe.close(); registry.evict(probe.getSessionId()); // CLI is installed, authenticated, and responsive } catch (IllegalStateException e) { throw new IllegalStateException("Claude CLI not available: " + e.getMessage(), e); } } } ``` ## Limitations `fork()` is **not yet implemented** — calling it throws `UnsupportedOperationException`. This will be added in a future release. * **Claude-only**: Sessions are currently only implemented for the Claude agent provider. Other providers will be added as their CLIs support persistent sessions. * **In-memory registry**: `ClaudeAgentSessionRegistry` stores sessions in a `ConcurrentHashMap`. Sessions do not survive application restarts — use `resume()` with a persisted session ID if you need cross-restart continuity. * **One working directory per session**: A session is anchored to the working directory specified at creation time. It cannot be changed. * **No context pruning**: Resumed sessions include the full conversation history. You cannot trim earlier turns to reduce token usage. ## Related * [Agent Client Overview](/projects/agent-client) — high-level project documentation * [Claude Agent SDK Tutorial: Session Resume](https://github.com/markpollack/claude-agent-sdk-java-tutorial/tree/main/module-11-session-resume) — lower-level SDK session resume * [Claude Agent SDK Tutorial: Session Fork](https://github.com/markpollack/claude-agent-sdk-java-tutorial/tree/main/module-12-session-fork) — lower-level SDK session fork # Lesson 1: Your First Agent Task Source: https://lab.pollack.ai/docs/agent-client/tutorial/01-first-task Create a file using an agent — set up, run, and verify `agent-client-tutorial/01-create-file` ## What You'll Build A Java program that asks an agent to create a file, then verifies the file was created. No Spring Boot required — just plain Java with the AgentClient API. ## Step 1: Set Up the Project Create a Maven project with the Claude agent dependency: ```xml theme={null} io.github.markpollack agent-claude 0.29.0 ``` Or clone the tutorial repo: ```bash theme={null} git clone https://github.com/markpollack/agent-client-tutorial.git cd agent-client-tutorial/01-create-file ``` ## Step 2: Understand the Code Three lines do the real work: ```java theme={null} // 1. Build the model — this is the provider-specific part ClaudeAgentModel model = ClaudeAgentModel.builder() .defaultOptions(ClaudeAgentOptions.builder() .model("claude-sonnet-4-5") .yolo(true) .build()) .build(); // 2. Create the client — this is the portable part AgentClient client = AgentClient.create(model); // 3. Run a goal AgentClientResponse response = client.run( "Create a file named hello.txt with the content 'Hello from Agent Client!'" ); ``` Notice the separation: * **Model construction** is provider-specific (you import `ClaudeAgentModel`) * **Client usage** is portable (`AgentClient.create()` and `.run()` work with any model) * **The goal** is a plain English string — describe *what*, not *how* ## Step 3: Run It ```bash theme={null} ./mvnw compile exec:java -Dexec.mainClass="HelloAgent" ``` ## Step 4: Verify Check that `hello.txt` was created: ```bash theme={null} cat hello.txt # Output: Hello from Agent Client! ``` ## What Just Happened 1. `ClaudeAgentModel` found the Claude CLI on your system 2. `AgentClient.create(model)` wrapped it in the portable client API 3. `.run(goal)` sent your English instruction to the Claude CLI 4. The CLI created the file, and the response told you it succeeded The agent did the same thing a developer would do — it created a file. The difference is you described *what* to do, not *how*. ## Next [Lesson 2: Multi-Provider →](/docs/agent-client/tutorial/02-multi-provider) — Run this same task with Codex and Gemini without changing the client code. # Lesson 2: Multi-Provider Source: https://lab.pollack.ai/docs/agent-client/tutorial/02-multi-provider Run the same task with Claude, Codex, and Gemini — the client code stays the same This lesson is doc-only — concepts apply to any tutorial module ## What You'll Learn How to run the exact same goal with three different providers. The model construction changes, but `AgentClient` usage is identical. ## The Pattern Provider-specific code is isolated to model construction. Everything after `AgentClient.create(model)` is portable: ```java theme={null} // Provider-specific: build the model AgentModel model = buildModel(provider); // Portable: same code regardless of provider AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run("Create hello.txt with 'Hello!'"); ``` ## Step 1: Run with Claude ```java theme={null} ClaudeAgentModel model = ClaudeAgentModel.builder() .defaultOptions(ClaudeAgentOptions.builder() .model("claude-sonnet-4-5") .yolo(true) .build()) .build(); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Claude!'" ); ``` ## Step 2: Run with Codex Swap the model — the client code is identical: ```java theme={null} CodexAgentModel model = new CodexAgentModel( CodexClient.create(), CodexAgentOptions.builder() .model("gpt-5-codex") .skipGitCheck(true) .build(), null ); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Codex!'" ); ``` `skipGitCheck(true)` lets Codex work in any directory. Without it, Codex requires a git repository. See [Codex Reference](/docs/agent-client/reference/codex-reference#skipgitcheck-and-mode-interaction) for details. ## Step 3: Run with Gemini ```java theme={null} GeminiAgentModel model = new GeminiAgentModel( GeminiClient.create(), GeminiAgentOptions.builder() .model("gemini-2.5-flash") .yolo(true) .build(), null ); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run( "Create a file named hello.txt with 'Hello from Gemini!'" ); ``` ## What Stayed the Same Across all three providers, these two lines are identical: ```java theme={null} AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run(goal); ``` The only difference is *which model you construct*. That's the portable API — `AgentClient` doesn't know or care which provider is behind it. ## Provider Differences While the client API is portable, providers have different capabilities: | Capability | Claude | Codex | Gemini | | ----------------- | ----------- | --------------------------- | ------------- | | Non-git directory | Works | Works (`skipGitCheck=true`) | Works | | Structured output | JSON Schema | Not supported | Not supported | | Session resume | Supported | Not supported | Not supported | | Permission modes | Multiple | full-auto only | yolo only | ## With Spring Boot In Spring Boot applications, you don't construct models manually. Use starter dependencies and Maven profiles to switch providers without changing code at all — see the [Switching Providers](/docs/agent-client/howto/switching-providers) how-to guide. ## Next Steps * [Switching Providers](/docs/agent-client/howto/switching-providers) — The Spring Boot profile pattern for zero-code provider switching * [Configuration Reference](/docs/agent-client/reference/portable-options) — All configuration options across providers * [Defaults Philosophy](/docs/agent-client/explanation/defaults-philosophy) — Why LOOSE mode exists and how it affects each provider # Lesson 3: Read and Transform Source: https://lab.pollack.ai/docs/agent-client/tutorial/03-read-and-transform Read files, perform data transformation, and write structured output `agent-client-tutorial/02-read-and-transform` ## What You'll Build A Spring Boot application that asks an agent to read log files, count severity levels (ERROR, WARNING, INFO), and produce a summary CSV. This demonstrates agents working with existing files and producing structured output. ## Step 1: Set Up Sample Data Clone the tutorial repository and navigate to the lesson: ```bash theme={null} git clone https://github.com/markpollack/agent-client-tutorial.git cd agent-client-tutorial/02-read-and-transform ``` The application creates sample log files automatically if they don't exist. Three service logs (`auth.log`, `api.log`, `worker.log`) with mixed severity levels. ## Step 2: Understand the Goal The goal is a multi-line prompt that specifies the input, the transformation, and the expected output format: ```java theme={null} String goal = """ In the logs/ directory there are multiple .log files from different services. Scan all .log files and count how many lines contain "ERROR", "WARNING", and "INFO". Output your results to summary.csv with the following structure: severity,count ERROR, WARNING, INFO, The output should be a valid CSV file with exactly four lines."""; ``` Notice the specificity: file format, column names, exact line count. Agents perform better with precise output specifications. ## Step 3: Run It ```bash theme={null} # With Claude (default) ./mvnw spring-boot:run # Or with a different provider ./mvnw spring-boot:run -Dspring.profiles.active=codex ``` ## Step 4: Verify Check the output CSV: ```bash theme={null} cat summary.csv # severity,count # ERROR,4 # WARNING,4 # INFO,7 ``` ## What's Different from Lesson 1 * **Agent reads existing files** — not just creating from scratch * **Multi-file input** — the agent scans a directory of logs * **Structured output** — the result is a well-defined CSV, not free text * **Data transformation** — counting, aggregating, formatting This pattern — read existing state, transform it, write structured output — is the core of most agent tasks in production. ## Next [Lesson 4: Git Operations →](/docs/agent-client/tutorial/04-git-operations) — Use an agent to inspect git history and recover lost changes. # Lesson 4: Git Operations Source: https://lab.pollack.ai/docs/agent-client/tutorial/04-git-operations Find lost git changes and merge them — agents working with version control `agent-client-tutorial/03-git-operations` ## What You'll Build An agent that inspects git history, finds commits on a detached HEAD, and merges them into the main branch. This demonstrates agents working with version control — a common real-world task. ## Step 1: Set Up the Repository Clone the tutorial and run the setup script that creates a git repo with "lost" changes: ```bash theme={null} git clone https://github.com/markpollack/agent-client-tutorial.git cd agent-client-tutorial/03-git-operations bash setup.sh ``` The setup script creates a git repository in `workspace/` with a commit on a detached HEAD — simulating changes that were accidentally "lost" when switching back to master. ## Step 2: Understand the Goal The prompt describes the problem in natural language, as a developer would: ```java theme={null} String goal = """ I just made some changes to my personal site and checked out master, but now I can't find those changes. Please help me find them and merge them into master."""; ``` The agent needs to: 1. Inspect the git reflog to find the detached commit 2. Identify the lost changes 3. Merge or cherry-pick them into master ## Step 3: Run It ```bash theme={null} ./mvnw spring-boot:run ``` The agent uses tools like `Bash` to run `git reflog`, `git log`, `git merge`, etc. ## Step 4: Verify ```bash theme={null} cd workspace git log --oneline -5 # Should show the recovered changes merged into master ``` ## What's Different from Previous Lessons * **Stateful environment** — the agent works in an existing git repository, not a clean directory * **Multi-step reasoning** — finding lost commits requires inspecting reflog, understanding branch state, and choosing a merge strategy * **Tool-heavy task** — the agent runs multiple shell commands, reads their output, and decides next steps * **STRICT mode applicable** — this task genuinely requires a git repository, making it a natural fit for `AgentClientMode.STRICT` ## The LOOSE/STRICT Connection Lessons 1-3 work in any directory — they don't need git. That's why LOOSE mode (the default) is appropriate. This lesson requires a git repository. If you were building a system that only runs git-aware tasks, STRICT mode would prevent accidental execution outside a git repo. See [Defaults Philosophy](/docs/agent-client/explanation/defaults-philosophy) for more on when to use each mode. ## Next Steps * [Getting Started](/docs/agent-client/howto/getting-started) — Recap the quick start for all three providers * [Switching Providers](/docs/agent-client/howto/switching-providers) — Run any tutorial lesson with a different provider * [Claude Reference](/docs/agent-client/reference/claude-reference) — Full configuration options including trace files # Agent Client Tutorial Source: https://lab.pollack.ai/docs/agent-client/tutorial/index Learn Agent Client step by step — from your first task to git operations Clone the repo and follow along: `git clone https://github.com/markpollack/agent-client-tutorial.git` ## Learning Path This tutorial takes you from zero to running real agent tasks. Each lesson builds on the previous one, progressing from simple file creation to multi-step git operations. Create a file using an agent — the simplest possible task. Set up a project, configure a provider, and verify the result. [Start Lesson 1 →](/docs/agent-client/tutorial/01-first-task) Run the same task with Claude, Codex, and Gemini. The client code stays the same — only the model construction changes. [Start Lesson 2 →](/docs/agent-client/tutorial/02-multi-provider) Read log files, count severity levels, and write a structured CSV summary. Agents working with existing files and producing structured output. [Start Lesson 3 →](/docs/agent-client/tutorial/03-read-and-transform) Find lost commits on a detached HEAD and merge them into master. Agents working with version control and multi-step reasoning. [Start Lesson 4 →](/docs/agent-client/tutorial/04-git-operations) ## Prerequisites * Java 17+ * Maven 3.9+ * At least one CLI agent installed: * Claude Code: `npm install -g @anthropic-ai/claude-code` * Codex CLI: `npm install -g @openai/codex` * Gemini CLI: `npm install -g @google/gemini-cli` ## Source Code All tutorial code is available in the [agent-client-tutorial](https://github.com/markpollack/agent-client-tutorial) repository: ```bash theme={null} git clone https://github.com/markpollack/agent-client-tutorial.git cd agent-client-tutorial ``` | Lesson | Directory | What it demonstrates | | ------ | ------------------------ | ---------------------------------------------------- | | 01 | `01-create-file/` | Simplest task — create a file | | 02 | (doc-only) | Multi-provider portability | | 03 | `02-read-and-transform/` | File reading, data transformation, structured output | | 04 | `03-git-operations/` | Git history inspection, merge recovery | # What's New Source: https://lab.pollack.ai/docs/agent-client/whats-new Release notes for Agent Client — auto-generated from git history ## 0.29.0 (2026-08-22) * Grok, Codex, and Antigravity now publish their parsed run trajectory through `AgentClientResponse.getPhaseCapture()`, matching the Claude facade contract; live facade integration gates verify that all three return a capture containing tool uses * Grok reads its native `streaming-json` ACP stream and Antigravity its `stream-json`; both SDKs retain the full stream for capture while preserving the existing terminal text, status, token, and session fields * Codex harvests its durable rollout JSONL after execution, locating the matching file by session id or working directory with a bounded wait for delayed flushes * The three provider adapters consume the released Agent Journal 1.8.0 capture modules; capture stays provider-owned, and `agent-model` and the production surface of `agent-client-core` remain free of journal dependencies * Repair LOOSE-mode Codex execution: current Codex versions reject `--full-auto` after `exec`, so full-auto now maps to the global `--sandbox workspace-write` and `--ask-for-approval never` options placed before the subcommand * Keep full-auto distinct from the explicit unrestricted `--dangerously-bypass-approvals-and-sandbox` level, so portable auto-approve no longer grants full-disk access implicitly; add repeated `--add-dir` support for explicitly named writable roots * Document that provider trajectories come from the `AgentClient` facade response rather than a direct `AgentApi.call()`, and are independent of raw trace-file configuration * Remove the inert module-local deploy skip from `agent-tck` — the artifact remains published — and delete the orphaned sandbox TCK package, which tested a sandbox SPI this repository no longer owns ## 0.28.0 (2026-08-22) * Add a Grok CLI provider: `agent-grok`, `grok-cli-sdk`, and `agent-starter-grok`. Headless runs return a real JSON envelope, so session id, token usage and a genuine per-run USD cost are read rather than scraped, and a caller-supplied session UUID makes resume possible without recovering an id from output * Add an Antigravity CLI provider for Google's `agy`: `agent-antigravity`, `antigravity-cli-sdk`, and `agent-starter-antigravity` * Drop `--effort` when the Antigravity model slug already encodes it; passing both is a hard error that returns no output, so a portable low-effort request would otherwise produce nothing at all * Declare the Antigravity working directory with `--add-dir`: setting the process working directory alone lets the CLI divert writes to a shared scratch directory while reporting success * Derive Antigravity run success from whether work was produced and nothing was refused, rather than from its `status` field, which reports ERROR alongside complete and correct responses * Detect Antigravity soft denials from both the result envelope and stderr, so a run whose tool calls were refused is not mistaken for one that found nothing to do * Add `GROK` and `ANTIGRAVITY` to the provider parity TCK and enrol both in the parity scenarios * Both new providers pass the full provider parity TCK — ten scenarios each, zero skips, zero failures — against live CLIs, joining Claude Code, Codex, and Gemini as the only adapters with parity coverage. CI re-verifies those three on every commit; Grok and Antigravity cannot run there because both CLIs authenticate interactively and cache credentials rather than reading an API key ## 0.27.0 (2026-08-21) * Repair published no-BOM consumer graphs with direct Jackson and Log4j declarations that survive flattened child POMs * Add one generated CI gate that discovers and resolves all 23 published runtime modules as fresh parentless, no-BOM consumers * Adopt Claude SDK 1.5.0, Agent Journal and Capture 1.7.0, and Agent Sandbox Core 0.10.0 while retaining the Java 21 contract * Produce one root CycloneDX 1.6 JSON SBOM and keep authoritative offline vulnerability analysis on the actual consumer JAR closures * Align current owned source headers and binary/source/Javadoc archive payloads with BSL 1.1 while retaining Apache history * Preserve immutable 0.26.0 as released; its Maven Central artifacts were not rebuilt or repaired ## 0.26.0 (2026-08-17) * Remove the experimental Vendir context advisor and Git-repository DSL as an intentional breaking change * Retire the stale `agents-runtime` container build, Docker integration TCK, and unused Docker dependencies * Keep Claude Code, Codex, and Gemini CLI as the actively verified provider set; classify the remaining adapters as experimental * Upgrade source-reactor Jackson and Log4j management; later public-consumer reproduction showed that flattened child POMs did not preserve every reviewed floor, which 0.27.0 corrects * Pin owned release automation and complete the repository-specific BSL 1.1 license text while preserving the historical Apache record ## 0.22.0 (2026-06-14) * agent-client 0.22.0: Spring AI 2.0.0 GA + Spring Boot 4.0.7; declare Jackson 2 explicitly * Upgrade default Gemini model to gemini-3.5-flash * Default Gemini model to gemini-3-flash-preview * Stop tracking generated .flattened-pom.xml; let parity runs finish ## 0.21.0 (2026-06-07) * Trust the workspace when Gemini yolo mode is enabled * Fix parity workflow JDK: project targets Java 21 * Fix agents-runtime Maven download: bump to 3.9.16 with archive fallback * Make Provider Parity real; add Codex to agents-runtime image; weekly CLI canary * Fix CLITransportCommandTest on machines without a Codex CLI * Add reasoning effort option: portable plus provider-native ## 0.20.0 (2026-06-06) * Step 3.1-3.3: Trace content mode, transcript archival, capture IT upgrades ## 0.19.0 (2026-05-28) * Verify traceDir in journal capture integration test * Add traceDir parser-level and uniqueness tests * Add trace-dir Spring Boot property and autoconfiguration * Add traceDir support to ClaudeAgentModel ## 0.18.0 (2026-05-15) * Remove judge-dependent modules from AgentClient * Update agent-journal deps to 1.1.0-SNAPSHOT * Skip tests on snapshot publish — CI handles test validation * Add agent-clis and API key secrets to CI and snapshot workflows * Migrate to markpollack org: package rename, BSL license, standalone POM, build-tools workflows ## 0.16.0 (2026-04-30) * Fix autoconfigure parent relativePath, remove plans from tracking * Add session journal for 2026-04-29 * Finalize roadmap with cross-framework stage and handoff notes * Add deprecated property migrator and update documentation * Rename property prefix from spring.ai.agents to agent-client * Introduce AgentApi interface, deprecate AgentModel * Extract AgentClientAutoConfiguration into separate module * Add cross-framework portability design document * Complete Stage 4 consolidation * Record LOOSE permission discoveries and add CLI validation to plan * Fix Gemini working directory not propagated to CLI process * LOOSE mode: bypass Codex sandbox, update model to gpt-5.4-mini * Reframe Stage 4 as LOOSE permission discovery via terminal-bench * Add property prefix rename to development plan * Update README with current docs, providers, and configuration ## 0.15.0 (2026-04-29) * Remove samples directory — covered by tutorial repo and external projects * Fix ClaudeProviderParityIT compilation error * Fix sample SNAPSHOT reference blocking release * Update project documentation and learnings * Add docgen tool and reference documentation infrastructure * Consolidate multi-provider sample and tutorial artifacts * Consolidate parity TCK and mode system infrastructure * Add provider parity CI matrix workflow * Wire ProviderParityTCK into Claude, Codex, Gemini IT modules * Add ProviderParityTCK with @ProviderCapability annotation * feat(codex): add AgentClientMode and default skipGitCheck to true ## 0.14.0 (2026-04-28) * Expose validate-commits as overridable release input * Remove formatPrompt boilerplate wrapping — pass goal verbatim * Add portable maxTurns, autoApprove, systemInstructions to AgentOptions ## 0.13.0 (2026-04-17) * Add portable jsonSchema support to AgentClient request spec * Add jsonSchema to portable AgentOptions interface ## 0.12.2 (2026-04-11) * fix(codex): add sandbox bypass flag and graceful error handling ## 0.12.1 (2026-04-10) * fix(codex): fix --model flag ordering and working directory propagation ## 0.12.0 (2026-04-10) * feat(qwen-code): add Qwen Code agent model and starter * feat(codex): surface outputSchema in CodexAgentOptions (#27) * fix(gemini,codex): fix response parsing and CLI compatibility (#29, #30) * Update README docs links to mintlify documentation site * feat(agent-claude): add ClaudeAgentSession and ClaudeAgentSessionRegistry * feat(agent-model): add AgentSession and AgentSessionRegistry interfaces * Update README.md ## 0.11.0 (2026-03-30) * Rename all project artifactIds: drop spring-ai- prefix * Rename spring-ai-agents-core to spring-ai-agent-launcher * Clean up dead module references and update README agent list * Update README with released 0.10.0 Maven coordinates ## 0.10.0 (2026-03-30) * Initial release. # What's New Source: https://lab.pollack.ai/docs/agent-experiment/whats-new Release notes for Agent Experiment ## 0.6.0 (2026-08-18) * Adds the `experiment-workflow` module for typed Agent Workflow invocation with journaled step cost/token evidence adapted to `AgentInvoker`. * Makes Agent Journal capture a first-class per-item lifecycle owned by `AgentExperiment`, using Agent Journal 1.7.0 and preserving variant, item, model, session, tool-use, and derived cost data. * Migrates runtime evaluation to Agent Judge 0.14.0 and stores normalized, Agent Experiment-owned judgments and verdicts. Existing 0.5 / Judge 0.13 results load automatically; re-saving writes the normalized format with the documented lossy boundary for obsolete metadata. * Updates to Agent Client 0.26.0, Agent Workflow 0.10.0, and Claude Code SDK 1.4.0. * Gives standalone consumers Jackson 2.21.6 and Jackson 3.1.6 safety floors without requiring a BOM; public 25-, 26-, and 57-JAR module closures have zero findings against the validated scan snapshot. * Publishes signed binary, source, and Javadoc artifacts plus one aggregate CycloneDX 1.6 JSON SBOM. * Requires Java 21. The credential-free reactor passed 544 tests; paid live Claude execution was not part of the release gate. ## 0.5.0 (2026-06-06) * Jackson 2.21.2, member pins (sdk 1.3.0, capture 1.4.0, judge 0.12.0), release 0.5.0 * Collect status: 2026-06-03 * Brief: abstain-aware consensus in judge aggregation ## 0.4.0 (2026-05-29) * Update dependencies to released versions for 0.4.0 release * Relax git dirty check to warn instead of throw for non-critical paths * Bump dependencies to next SNAPSHOT versions ## 0.3.0 (2026-05-28) * Update dependencies to released versions for 0.3.0 release * Fix RunLogManager forcing logback on all experiment-core consumers * Bump dependencies to next SNAPSHOT versions ## 0.2.0 (2026-05-19) * Update dependencies to released versions for 0.2.0 release * Bump agent-judge dependency from 0.10.0-SNAPSHOT to 0.10.0 * Step 3.4: Final consolidation — all stages complete * Step 3.3: Jackson polymorphic serialization for both ExecutionDetail subtypes * Step 3.2: Add JudgeExperiment and JudgeExperimentResult * Step 3.1: Add JudgeScorer, supporting types, and JudgeExecutionDetail * Step 3.0: Stage 3 entry — verify ExecutionDetail supports JudgeExecutionDetail * Step 2.3: Stage 2 consolidation — verify re-eval→compare pipeline * Step 2.2: Add ReEvaluator for post-hoc re-scoring without re-invocation * Step 2.1: Add ReEvaluationContextFactory and AgentReEvaluationContextFactory * Update ROADMAP.md checkboxes for Step 2.0 * Step 2.0: Stage 2 entry — verify prerequisites, document scope adjustments * Migrate dependencies from org.springaicommunity to io.github.markpollack * Step 1.3: Stage 1 consolidation — verify boundaries, compact learnings * Step 1.2: Rename ExperimentRunner to AgentExperiment * Ignore plans scratch files (META-ROADMAP.md, status.md) * Step 1.1: Decouple ItemResult from InvocationResult via ExecutionDetail * Step 1.0: Design review — map types to shared vs agent-specific boundaries * Ignore entire plans/ directory in .gitignore ## 0.1.0 (2026-03-30) * Initial release. # What's New Source: https://lab.pollack.ai/docs/agent-hooks/whats-new Release notes for Agent Hooks ## 0.7.0 (2026-08-19) * **Standalone consumers get safe Jackson versions.** A consumer that resolved `agent-hooks-spring` or `agent-hooks-gemini` without importing a BOM previously selected Jackson versions with known advisories, because the parent's `dependencyManagement` does not travel through the flattened POM published to Maven Central. The floors are now 2.21.6 and 3.1.6, with the Jackson 3 floor declared directly on `agent-hooks-spring`. * **Published POMs no longer carry repositories.** 0.6.x shipped a `` block on every module POM, so consumers inherited two snapshot repositories and a milestone repository. The build resolves from Maven Central alone. * **`agent-hooks-claude` is Java 21.** Every published `claude-code-sdk` version is Java 21 bytecode, so the module never could be used on Java 17; it had been shipping as Java 17 bytecode. `agent-hooks-core`, `-spring`, and `-gemini` remain Java 17. * Claude Agent SDK moves to 1.5.0 (still `provided` scope). An actual no-BOM consumer of Agent Hooks plus that supplied SDK resolves the corrected Jackson 2.21.6 and 3.1.6 floors. * Every distributed archive now contains the BSL 1.1 license text. * The parent artifact publishes one aggregate CycloneDX 1.6 SBOM; child modules publish none. * A hook that throws, and a steering decision returned from an observation-only event, are now logged; both were documented as logged and were in fact silent. * The Gemini dispatcher writes a protocol response for every invocation, so an unchecked hook failure can no longer leave the CLI waiting on empty stdout. * Adapter log and exception messages no longer embed tool-input payloads. * `AgentHookBridge` gains `evictSession(String)` and `activeSessionCount()`. ## 0.6.4 (2026-06-15) * Spring AI 2.0.0 GA and Spring Boot 4.0.7 ## 0.6.3 (2026-06-06) * Add .flattened-pom.xml to .gitignore * Prepare for next development iteration 0.6.3-SNAPSHOT ## 0.6.2 (2026-04-10) * Fix stale javadoc reference to removed AgentHookEvent enum * Step 6.K: Consolidation — docs, quality, landscape fix * Step 6.3: Cross-adapter proof — 3 runtimes * Step 6.2: GeminiHookDispatcher + GeminiDecisionMapper * Step 6.1: Gemini module skeleton + 7 event types * Stop tracking CLAUDE.md and plans/ — internal-only files * Stage 5: Claude Agent SDK adapter — write once, run on both runtimes ## 0.5.0 (2026-04-09) * Initial release. # Analyze Runs with DuckDB Source: https://lab.pollack.ai/docs/agent-journal/analyzing-runs Query Agent Journal 1.8.0 event streams with executable DuckDB examples `JsonFileStorage` writes one `events.jsonl` file per run. DuckDB can read all runs for an experiment directly; no import step or fixed table schema is required. The examples below assume this path: ```text theme={null} .agent-journal/experiments/{experiment-id}/runs/{run-id}/events.jsonl ``` Pass `union_by_name = true` because different event types have different fields. Quote `"@type"` because the discriminator contains `@`. Header rows have `@type = 'header'`, so event filters exclude them. ## Count tool calls ```sql theme={null} SELECT toolName AS tool, COUNT(*) AS calls, ROUND(AVG(durationMs), 0) AS average_ms, COUNT(*) FILTER (WHERE NOT success) AS failures FROM read_ndjson_auto( '.agent-journal/experiments/*/runs/*/events.jsonl', union_by_name = true) WHERE "@type" = 'tool_call' GROUP BY toolName ORDER BY calls DESC, tool; ``` ## Summarize LLM usage and cost by run file Ask DuckDB to add the source filename explicitly: ```sql theme={null} SELECT filename, SUM(tokenUsage.inputTokens) AS input_tokens, SUM(tokenUsage.outputTokens) AS output_tokens, SUM( cost.inputCostUsd + cost.outputCostUsd + cost.thinkingCostUsd - cost.cacheSavingsUsd ) AS total_cost_usd FROM read_ndjson_auto( '.agent-journal/experiments/*/runs/*/events.jsonl', union_by_name = true, filename = true) WHERE "@type" = 'llm_call' GROUP BY filename ORDER BY filename; ``` ## Extract ordered tool sequences ```sql theme={null} SELECT regexp_extract(filename, 'runs/([^/]+)/events', 1) AS run_id, ROW_NUMBER() OVER ( PARTITION BY filename ORDER BY timestamp ) AS sequence_number, toolName AS tool FROM read_ndjson_auto( '.agent-journal/experiments/*/runs/*/events.jsonl', union_by_name = true, filename = true) WHERE "@type" = 'tool_call' ORDER BY run_id, sequence_number; ``` ## Count adjacent tool transitions ```sql theme={null} WITH tool_sequence AS ( SELECT filename, toolName AS tool, ROW_NUMBER() OVER ( PARTITION BY filename ORDER BY timestamp ) AS sequence_number FROM read_ndjson_auto( '.agent-journal/experiments/*/runs/*/events.jsonl', union_by_name = true, filename = true) WHERE "@type" = 'tool_call' ) SELECT current_step.tool AS from_tool, next_step.tool AS to_tool, COUNT(*) AS transitions FROM tool_sequence current_step JOIN tool_sequence next_step ON current_step.filename = next_step.filename AND next_step.sequence_number = current_step.sequence_number + 1 GROUP BY from_tool, to_tool ORDER BY transitions DESC, from_tool, to_tool; ``` Transition counts describe recorded behavior; they do not establish a quality threshold by themselves. Interpret them with task outcomes, run configuration, and enough repeated trials for the comparison you intend to make. ## Analyze derived events separately Per-step cost attribution and step outcomes live in `analysis.jsonl`, not `events.jsonl`. Read that file with the same `read_ndjson_auto(..., union_by_name = true)` pattern. Use the run directory or stream header for run identity, and join an analysis `stepId` to the execution event's stable `id` when the event has one. DuckDB reads the JSONL files directly, but Agent Journal's Java `loadEvents` and `loadDerivedEvents` methods currently load the requested file into memory. Use DuckDB or another streaming data tool for large cross-run analysis. # API Reference Source: https://lab.pollack.ai/docs/agent-journal/api-reference Agent Journal 1.8.0 core, storage, event, evaluation, and capture APIs This page describes the public API shipped in Agent Journal 1.8.0. ## Coordinates | Artifact | Purpose | Java | | ----------------------------------------------------- | ------------------------------------------------------ | ----------- | | `io.github.markpollack:journal-core:1.8.0` | Core domain, storage, evaluation, feedback, and traces | 17+ | | `io.github.markpollack:claude-code-capture:1.8.0` | Claude Code capture | 21+ runtime | | `io.github.markpollack:gemini-cli-capture:1.8.0` | Gemini CLI capture | 21+ runtime | | `io.github.markpollack:grok-cli-capture:1.8.0` | Grok CLI capture | 17+ | | `io.github.markpollack:codex-cli-capture:1.8.0` | Codex CLI capture | 17+ | | `io.github.markpollack:antigravity-cli-capture:1.8.0` | Antigravity CLI capture | 17+ | ## `Journal` | Method | Result | Notes | | ---------------------------------------------------------- | ---------------- | ------------------------------------------------------- | | `run(String experimentId)` | `RunBuilder` | Primary run factory | | `experiment(String experimentId)` | `Experiment` | Gets or creates an experiment | | `experiment(String id, Experiment.Builder builder)` | `Experiment` | Builder is ignored if the experiment already exists | | `configure(JournalStorage storage)` | `void` | Call before creating runs | | `storage()` | `JournalStorage` | Returns the global backend | | `registerEventType(String, Class)` | `void` | Registers an external subtype on the configured storage | | `reset()` | `void` | Resets storage configuration and the experiment cache | ## `RunBuilder` | Method | Purpose | | ------------------------ | ----------------------------------------------------------------- | | `name(String)` | Human-readable run name | | `agent(String)` | Agent identifier | | `task(String)` | Task identifier | | `repository(String)` | Accepts a repository path; 1.8.0 does not persist it or act on it | | `config(Config)` | Replace the run configuration | | `config(String, Object)` | Add one immutable input value | | `tags(Tags)` | Replace tags | | `tag(String, String)` | Add one tag | | `previousRun(String)` | Link a retry to the prior run ID | | `parentRun(String)` | Link a child run to a parent run ID | | `start()` | Create a `RUNNING` `Run` | There is no `config(Map)` overload in 1.8.0. The `repository(String)` setter is present, but the 1.8.0 run implementation does not consume the stored builder value. ## `Run` | Area | Methods | | ----------- | --------------------------------------------------------------------------------- | | Identity | `id()`, `name()`, `experiment()`, `agentId()`, `previousRunId()`, `parentRunId()` | | Data | `config()`, `summary()`, `tags()`, `status()` | | Execution | `logEvent(JournalEvent)`, `logMetric(...)` | | Analysis | `logDerivedEvent(DerivedEvent)` | | Artifacts | `logArtifact(String, String)`, `logArtifact(String, byte[], Map)` | | Instruments | `metrics()`, `calls()` | | Lifecycle | `setSummary(...)`, `fail(Throwable)`, `finish(RunStatus)`, `close()` | ## Event factories `JournalEvent` is extensible. Built-in event records include timestamps and expose their Java `type()` value; file serialization uses the registered `@type` name. | Event | Common factory or builder | | --------------------- | -------------------------------------------------------------------------------------------------- | | `LLMCallEvent` | `of(model, inputTokens, outputTokens, costUsd)`, `builder()` | | `ToolCallEvent` | `success(tool, input, output, durationMs)`, `failure(tool, input, error, durationMs)`, `builder()` | | `StateChangeEvent` | `of(from, to, reason)` | | `MetricEvent` | `of(name, value, tags)` | | `CustomEvent` | `of(name)`, `of(name, attributes)` | | `GitPatchEvent` | `of(baseBranch, fileChanges)` | | `GitCommitEvent` | `of(sha, message, branch)` | | `GitBranchEvent` | `created(branchName, fromRef)`, `checkedOut(branchName)`, `deleted(branchName)` | | `GitPullRequestEvent` | `created(...)`, `updated(...)`, `merged(...)`, `closed(...)` | ### Token, cost, and timing values ```java theme={null} TokenUsage simple = TokenUsage.of(1200, 450); TokenUsage withThinking = TokenUsage.of(1200, 450, 300); TokenUsage allFields = new TokenUsage( 1200, 450, 300, 200, 800, 0); CostBreakdown cost = CostBreakdown.of(0.018, 0.00675); TimingInfo timing = TimingInfo.of(2500, 2100, 400); ``` The `TokenUsage` constructor order is input, output, thinking, cache creation, cache read, and tool-use tokens. There is no five-argument `TokenUsage.of(...)` overload in 1.8.0. ## `JournalStorage` | Area | Operations | | -------------------- | ------------------------------------------------------------------------- | | Experiments | `saveExperiment`, `loadExperiment`, `listExperiments`, `experimentExists` | | Runs | `saveRun`, `loadRun`, `listRuns`, `runExists` | | Execution events | `appendEvent`, `loadEvents` | | Derived events | `appendDerivedEvent`, `loadDerivedEvents`, `persistsDerivedEvents` | | Feedback | `appendFeedback`, `loadFeedback` | | Artifacts | `saveArtifact`, `loadArtifact`, `listArtifacts` | | External event types | `registerEventSubtype` | `JsonFileStorage` implements every operation and reports durable derived-event persistence. `InMemoryStorage` implements every operation in memory and reports the default non-durable derived-event behavior. ## Evaluation API `EvalSubjectSources.fromJournal(storage, experimentId, runId)` reads a run and produces subjects. `EvalSubjectSources.fromEvents(events, runId)` adapts an existing event list. `EvalSubjectQuery` supports: | Method | Purpose | | ------------------------------- | ------------------------------- | | `from(EvalSubjectSource)` | Start a query | | `kind(EvalSubjectKind)` | Filter by kind | | `where(Predicate)` | Add a predicate | | `toSet()` | Materialize an `EvalSubjectSet` | | `groupBy(Function)` | Group into subject sets | | `countBy(Function)` | Count by classifier | | `count()` | Count matches | ## Capture parsers ### Claude Code ```java theme={null} SessionLogParser.parse(response, phaseName, promptText) SessionLogParser.parse(response, phaseName, promptText, traceFile) SessionLogParser.parse(response, phaseName, promptText, traceFile, contentMode) SessionLogParser.parse(response, phaseName, promptText, traceFile, contentMode, rawMode) ``` `response` is an `Iterator`. The default content mode for trace-writing overloads is `TRUNCATED`; the default raw mode is `NONE`. `RunRecorder` wraps an open `Run`, inherits `recordPhase(PhaseCapture)`, exposes `run()` and `lenient()`, and owns `finish()`/`close()`. ### Gemini CLI ```java theme={null} GeminiSessionParser.parse(result, phaseName, promptText) GeminiSessionParser.parse(result, phaseName, promptText, traceFile) GeminiSessionParser.parse(result, phaseName, promptText, traceFile, contentMode) ``` `result` is a Gemini SDK `QueryResult`. `GeminiRunRecorder.recordPhase(GeminiPhaseCapture)` records the core and derived projections. ## Storage cautions * `JsonFileStorage` does not validate path containment for caller-supplied identifiers or artifact names in 1.8.0. * Use one writer per run and do not assume multi-process safety. * File-backed load methods read the requested JSONL file into memory. * Portable traces can contain sensitive content; see [Capture SDK Sessions](/docs/agent-journal/capture-sessions). # Capture SDK Sessions Source: https://lab.pollack.ai/docs/agent-journal/capture-sessions Record Claude Code, Gemini CLI, Grok, Codex, and Antigravity results with Agent Journal 1.8.0 Agent Journal provides five vendor adapters that write compatible core events and portable traces. The Claude and Gemini adapters parse a typed SDK object: Claude consumes the [Claude Agent SDK for Java](/projects/claude-agent-sdk) and Gemini the `gemini-cli-sdk` published with [Agent Client](/projects/agent-client). Both require a Java 21 runtime, because those SDK dependencies are published as Java 21 bytecode. The Grok, Codex, and Antigravity adapters parse their CLI's durable JSONL directly and carry no vendor SDK dependency, so they run on Java 17 like `journal-core`. All three produce an ordered tool trajectory; Gemini does not. ## Claude Code Add the adapter: ```xml theme={null} io.github.markpollack claude-code-capture 1.8.0 ``` `SessionLogParser` converts an `Iterator` from the Claude Code SDK into a `PhaseCapture`. The six-argument overload also writes a portable trace and controls modeled and raw content independently: ```java theme={null} import io.github.markpollack.claude.agent.sdk.parsing.ParsedMessage; import io.github.markpollack.journal.claude.PhaseCapture; import io.github.markpollack.journal.claude.SessionLogParser; import io.github.markpollack.journal.trace.TraceContentMode; import io.github.markpollack.journal.trace.TraceRawMode; import java.nio.file.Path; import java.util.Iterator; PhaseCapture phase = SessionLogParser.parse( response, "implement", promptText, Path.of(".agent-journal/traces/implement.jsonl"), TraceContentMode.TRUNCATED, TraceRawMode.NONE); ``` Here, `response` is an `Iterator` returned by the SDK and `promptText` is the exact prompt or `null`. Record the capture into a durable journal with `RunRecorder`: ```java theme={null} import io.github.markpollack.journal.Journal; import io.github.markpollack.journal.claude.RunRecorder; import io.github.markpollack.journal.storage.JsonFileStorage; import java.nio.file.Path; Journal.configure(new JsonFileStorage(Path.of(".agent-journal"))); try (RunRecorder recorder = new RunRecorder( Journal.run("capture-experiment").agent("claude-code").start())) { recorder.recordPhase(phase); } ``` The recorder writes execution events to `events.jsonl` and per-step cost attribution to `analysis.jsonl`. It fails at finish when it produced derived events but the configured backend does not persist them durably. Call `lenient()` only when losing the derived stream after process exit is intentional. ## Gemini CLI Add the adapter: ```xml theme={null} io.github.markpollack gemini-cli-capture 1.8.0 ``` `GeminiSessionParser` converts the Gemini SDK's synchronous `QueryResult` into a `GeminiPhaseCapture`: ```java theme={null} import io.github.markpollack.agents.geminisdk.types.QueryResult; import io.github.markpollack.journal.Run; import io.github.markpollack.journal.gemini.GeminiPhaseCapture; import io.github.markpollack.journal.gemini.GeminiRunRecorder; import io.github.markpollack.journal.gemini.GeminiSessionParser; GeminiPhaseCapture phase = GeminiSessionParser.parse( result, "query", "summarize the repository"); new GeminiRunRecorder(run).recordPhase(phase); ``` Here, `result` is a `QueryResult` and `run` is an open Agent Journal `Run`. Configure `JsonFileStorage` if the derived `StepCostEvent` must survive process exit. Gemini's typed SDK exposes result-level text, usage, cost, duration, model, and status, but it does not expose per-tool calls. The adapter therefore records one turn-level step and does not invent tool detail. It has no `TraceRawMode` because the typed Gemini message does not retain a verbatim wire envelope. ## Grok CLI Add the adapter: ```xml theme={null} io.github.markpollack grok-cli-capture 1.8.0 ``` `GrokSessionParser` reads Grok's ACP-shaped `streaming-json` output from a file or a `BufferedReader`: ```java theme={null} import io.github.markpollack.journal.Run; import io.github.markpollack.journal.grok.GrokPhaseCapture; import io.github.markpollack.journal.grok.GrokRunRecorder; import io.github.markpollack.journal.grok.GrokSessionParser; import java.nio.file.Path; GrokPhaseCapture phase = GrokSessionParser.parse( Path.of("grok-stream.jsonl"), "implement", promptText); new GrokRunRecorder(run).recordPhase(phase); ``` The parser pairs `tool_call` and `tool_call_update` records by `toolCallId`, keeping both the tool name Grok reports and its ACP semantic kind. Grok's stream carries a real session cost in `end.total_cost_usd` but no durable join from a turn to the tools it ran, so the session total is retained and attributed evenly across the captured tool steps. ## Codex CLI Add the adapter: ```xml theme={null} io.github.markpollack codex-cli-capture 1.8.0 ``` `CodexSessionParser` reads a durable Codex rollout JSONL file: ```java theme={null} import io.github.markpollack.journal.Run; import io.github.markpollack.journal.codex.CodexPhaseCapture; import io.github.markpollack.journal.codex.CodexRunRecorder; import io.github.markpollack.journal.codex.CodexSessionParser; import java.nio.file.Path; CodexPhaseCapture phase = CodexSessionParser.parse( Path.of("rollout.jsonl"), "implement", promptText); new CodexRunRecorder(run).recordPhase(phase); ``` Codex records most tool invocations under the outer function name `exec`, which would collapse every step into one undifferentiated name. The parser deliberately classifies the nested `payload.input` call instead, extracting `exec_command` arguments without evaluating them and assigning semantic names such as `Search`, `Read`, `Inspect`, `Test`, `Build`, and `Git`. The classifier is conservative about shell syntax. Aliases, shell functions, quoted operators, and work hidden behind an interpreter fall back to `Shell`. The raw input and command are retained on the record, so callers can reclassify them later. Codex rollout token counts, including cached and reasoning tokens, are preserved. ## Antigravity CLI Add the adapter: ```xml theme={null} io.github.markpollack antigravity-cli-capture 1.8.0 ``` `AntigravitySessionParser` reads Antigravity's streaming JSON: ```java theme={null} import io.github.markpollack.journal.Run; import io.github.markpollack.journal.antigravity.AntigravityPhaseCapture; import io.github.markpollack.journal.antigravity.AntigravityRunRecorder; import io.github.markpollack.journal.antigravity.AntigravitySessionParser; import java.nio.file.Path; AntigravityPhaseCapture phase = AntigravitySessionParser.parse( Path.of("antigravity-stream.jsonl"), "implement", promptText); new AntigravityRunRecorder(run).recordPhase(phase); ``` The parser pairs active and terminal updates by `step_index` and handles both `DONE` and `ERROR` steps, recording parameters, output or error text, duration, and terminal token usage. Stable identities are derived from the conversation and step positions. ## Cost provenance Neither Codex nor Antigravity reports monetary cost in these streams. Their cost records are zero-valued and explicitly marked `costAvailable=false` with an `unreported` source, so a downstream reader can distinguish "this run cost nothing" from "this CLI never said." ## Portable trace modes `TraceContentMode` controls content bodies on modeled trace lines: | Mode | Modeled content behavior | | ----------- | ---------------------------------------------------------------------- | | `FULL` | Writes complete assistant text, thinking text, and tool-result content | | `TRUNCATED` | Default; caps each modeled content body at 60,000 characters | | `LENGTHS` | Omits those modeled content bodies and records their lengths | `TraceRawMode` is Claude-only and orthogonal: | Mode | Raw vendor messages | | ------ | ---------------------------------------------------------------- | | `NONE` | Default; no raw vendor-message lines | | `FULL` | Writes each available vendor message as an unredacted `raw` line | All trace modes require a data-handling decision. The capture record and journal events can contain the caller-supplied prompt text. Tool inputs are present on portable `tool_use` lines even in `LENGTHS` mode. `TRUNCATED` and `FULL` portable traces can include assistant text, file contents, tool results, and command output. `TraceRawMode.FULL` adds the complete available vendor message and can include fields omitted from the typed model. Treat trace and journal files as sensitive, restrict filesystem access, and keep them out of version control. ## Two schemas Do not conflate the two JSONL formats: * Canonical journal streams use `@type` and currently have schema version 1. * Portable capture traces use `type`, `ts`, and `seq` and currently have schema version 2. Both formats have their own header and evolution contract. # Core Concepts Source: https://lab.pollack.ai/docs/agent-journal/concepts Experiments, runs, event streams, derived analysis, feedback, and storage boundaries Agent Journal models a bounded execution as a `Run` and groups comparable runs in an `Experiment`. ```text theme={null} Experiment └── Run ├── Config immutable inputs ├── events.jsonl recorded execution facts ├── analysis.jsonl derived interpretations ├── feedback.jsonl reviewer feedback ├── Summary mutable outputs in run.json └── Artifacts caller-named byte content ``` ## Experiments and runs An experiment is a stable grouping identifier for repeated trials. A run is one execution with a start time, an optional finish time, and a `RUNNING`, `FINISHED`, or `FAILED` status. Inputs belong in `Config`, which becomes immutable when the run starts. Outputs belong in `Summary`, where the latest value for a key wins. Runs can link to a prior attempt with `previousRun(...)` or to a parent execution with `parentRun(...)`. These are identifiers recorded on the run; Agent Journal does not schedule retries or child agents. ## Execution events `JournalEvent` is an extensible interface, not a closed or sealed hierarchy. Agent Journal 1.8.0 registers these built-in JSON subtypes: | `@type` | Java type | Purpose | | -------------- | --------------------- | ------------------------------------------------------------------- | | `llm_call` | `LLMCallEvent` | Provider, model, token, cost, timing, and response metadata | | `tool_call` | `ToolCallEvent` | Tool name, input, output or error, duration, and optional stable ID | | `state_change` | `StateChangeEvent` | Named state transition and reason | | `metric` | `MetricEvent` | Point-in-time numeric measurement with tags | | `custom` | `CustomEvent` | Application-defined attributes | | `git_patch` | `GitPatchEvent` | File-level patch summary | | `git_commit` | `GitCommitEvent` | Commit identity and metadata | | `git_branch` | `GitBranchEvent` | Branch operation | | `git_pr` | `GitPullRequestEvent` | Pull request operation | | `feedback` | `FeedbackEvent` | Human feedback when serialized as an event | Register an external event implementation before reading it from file storage: ```java theme={null} Journal.registerEventType("workflow_step", WorkflowStepEvent.class); ``` The type name and class are supplied by the integration that owns the external event. ## Recorded and derived data `events.jsonl` is the append-only record of what the application logged during execution. `analysis.jsonl` is a separate append-only stream for interpretations computed about that execution. Agent Journal 1.8.0 provides two derived event types: * `StepCostEvent` records a cost allocation and preserves the actual run cost separately from the attributed share. * `StepOutcomeEvent` records caller-supplied goal distance and outcome metrics for a step. Both streams begin with an independent schema header when the first event is appended. `JsonFileStorage` skips header lines when loading events. The canonical stream schema version is independent from the portable trace schema version. ## Metrics and calls Every run exposes a `MetricRegistry` with counters, timers, and gauges. `run.logMetric(...)` also appends a `MetricEvent` to the execution stream. `CallTracker` records an in-memory hierarchy of named operations and durations within a run. It does not create distributed spans, a collector, or a monitoring service. ## Evaluation subjects and feedback `EvalSubjectSources` converts selected journal events into source-neutral subjects for evaluation. The core adapter maps LLM calls, tool calls, state changes, and custom events; metric and git events are skipped. The `EvalSubjectKind` enum also reserves kinds used by other adapters, such as workflow steps, router decisions, retrieval results, final outputs, and feedback. The feedback API stores reviewer judgments in `feedback.jsonl`. It supports binary, numerical, and categorical scores and can export reviewed items for labeled datasets. ## Storage behavior | Behavior | `InMemoryStorage` | `JsonFileStorage` | | ----------------- | ------------------------ | ----------------------------------- | | Survives JVM exit | No | Yes | | Execution events | In memory | `events.jsonl` | | Derived events | In memory only | `analysis.jsonl` | | Feedback | In memory | `feedback.jsonl` | | Artifacts | Cloned byte arrays | Files under `artifacts/` | | Intended use | Tests and ephemeral work | Local development and research runs | `JsonFileStorage` appends one JSON object per line without rewriting the existing stream. Its load methods currently read the whole requested file into memory before deserializing it. ## Operational and security boundaries `JsonFileStorage` is a local filesystem backend, not a database, access-control layer, or multi-tenant security boundary. * Treat journal directories as potentially sensitive operational data. * Use trusted caller-controlled experiment IDs, run IDs, and artifact names; 1.8.0 does not enforce path containment. * Use one writer per run for file-backed storage. * Do not infer concurrent-append or multi-process safety from the append-only format; current tests do not establish those guarantees. * Large journals can require substantial heap when loaded because file-backed reads use whole-file loading. For content captured from vendor SDKs, see [Capture SDK Sessions](/docs/agent-journal/capture-sessions). # Getting Started Source: https://lab.pollack.ai/docs/agent-journal/getting-started Record and persist a first Agent Journal run with journal-core 1.8.0 This tutorial records one LLM call and one tool call, then persists the run to local JSON and JSONL files. ## Requirements * Java 17 or later for `journal-core` * Maven 3.9 or the Maven Wrapper ## 1. Add `journal-core` ```xml theme={null} io.github.markpollack journal-core 1.8.0 ``` ## 2. Configure local storage Configure storage before creating a run: ```java theme={null} import io.github.markpollack.journal.Journal; import io.github.markpollack.journal.storage.JsonFileStorage; import java.nio.file.Path; Journal.configure(new JsonFileStorage(Path.of(".agent-journal"))); ``` The default backend is `InMemoryStorage`, so a run is not durable unless you configure storage. ## 3. Record a run ```java theme={null} import io.github.markpollack.journal.Journal; import io.github.markpollack.journal.Run; import io.github.markpollack.journal.event.LLMCallEvent; import io.github.markpollack.journal.event.ToolCallEvent; import java.util.Map; try (Run run = Journal.run("first-experiment") .name("attempt-1") .agent("example-agent") .task("summarize-repository") .config("model", "claude-opus-4-5") .tag("environment", "local") .start()) { run.logEvent(LLMCallEvent.of("claude-opus-4-5", 1200, 450, 0.02475)); run.logEvent(ToolCallEvent.success( "Bash", Map.of("command", "git status --short"), Map.of("exitCode", 0, "output", ""), 150)); run.setSummary("success", true); run.logArtifact("result.txt", "Repository is clean."); } ``` `Run` implements `AutoCloseable`. Normal closure produces `FINISHED`; call `run.fail(exception)` to record `FAILED` explicitly. ## 4. Inspect the files The example produces this layout: ```text theme={null} .agent-journal/ └── experiments/ └── first-experiment/ ├── experiment.json └── runs/ └── {run-id}/ ├── run.json ├── events.jsonl └── artifacts/ └── result.txt ``` The first `events.jsonl` line is a schema header. Each following line is one event and carries an `@type` discriminator. ```json theme={null} {"@type":"header","schemaVersion":1,"stream":"events","runId":"..."} {"@type":"llm_call","timestamp":"...","provider":null,"model":"claude-opus-4-5","tokenUsage":{"inputTokens":1200,"outputTokens":450,"thinkingTokens":0,"cacheCreationTokens":0,"cacheReadTokens":0,"toolUseTokens":0},"cost":{"inputCostUsd":0.02475,"outputCostUsd":0.0,"thinkingCostUsd":0.0,"cacheSavingsUsd":0.0},"timing":null,"finishReason":null,"responseId":null,"metadata":{}} ``` Use caller-controlled trusted identifiers for `experimentId`, `runId`, and artifact names. Version 1.8.0 does not enforce path containment for those values, so do not pass untrusted path fragments or separators. ## 5. Isolate tests Use the in-memory backend and reset global state around each test: ```java theme={null} import io.github.markpollack.journal.Journal; import io.github.markpollack.journal.storage.InMemoryStorage; Journal.reset(); Journal.configure(new InMemoryStorage()); try { try (var run = Journal.run("test-experiment").start()) { run.setSummary("result", "ok"); } } finally { Journal.reset(); } ``` ## Next steps Learn what is persisted and what is derived. Query the generated event files with DuckDB. # What's New Source: https://lab.pollack.ai/docs/agent-journal/whats-new Release notes for Agent Journal — auto-generated from git history ## 1.8.0 (2026-08-22) * Three new tool-trajectory capture adapters — `grok-cli-capture`, `codex-cli-capture`, and `antigravity-cli-capture` — turn each CLI's durable JSONL stream into ordered `ToolCallEvent` and `StepCostEvent` records with stable tool-call identities * Grok capture parses the ACP-shaped `streaming-json` output, pairing `tool_call` and `tool_call_update` by `toolCallId` and preserving both the reported tool name and the ACP semantic kind. The stream has no durable turn-to-tool cost join, so the real session total from `end.total_cost_usd` is retained and attributed evenly across captured tool steps * Codex capture parses durable rollout JSONL and deliberately classifies the nested `payload.input` call rather than the outer `exec` function name that Codex records for most tool invocations, assigning semantic names such as `Search`, `Read`, `Inspect`, `Test`, `Build`, and `Git`. The classifier is conservative around shell syntax — aliases, shell functions, quoted operators, and work hidden behind an interpreter fall back to `Shell`, with raw input and command retained for later reclassification * Antigravity capture pairs active and terminal updates by `step_index`, handles both `DONE` and `ERROR` steps, records parameters, output or error text, duration, and terminal token usage, and derives stable identities from conversation and step positions * Codex and Antigravity report no monetary cost in these streams, so cost records are zero-valued and explicitly marked `costAvailable=false` with an `unreported` source * Gemini CLI capture remains turn-level and provides no tool trajectory, so Gemini is not part of this release's multi-CLI tool-trajectory claim * The Claude Code SDK dependency moves from 1.4.0 to 1.5.0 * Additive on the existing event and trace contracts. `journal-core` and the three new capture modules target Java 17; the Claude Code and Gemini CLI capture modules still require Java 21 for their vendor SDKs. 488 tests pass across the seven-module reactor ## 1.7.0 (2026-08-18) * Standalone consumers now resolve CVE-clear Jackson floors without an external BOM: Jackson 2.21.6 for all three modules and Jackson 3.1.6 on Claude capture's transitive Jackson 3 path. Agent Journal's source and stored formats continue to use Jackson 2; this is not a Jackson 3 migration. * Every binary and source JAR now includes `META-INF/LICENSE`, and every Javadoc JAR includes `resources/LICENSE`. The BSL 1.1 terms are unchanged. * The parent artifact publishes an aggregate CycloneDX 1.6 JSON SBOM for all three modules under the `cyclonedx` classifier. * Capture-module POMs now declare their actual Java 21 runtime requirement; `journal-core` remains Java 17. * Snapshot publishing now runs the full test suite, and shared build/release workflows are pinned to an immutable commit. * No API, event-schema, or trace-format change: both journal streams remain schema version 1 and portable traces remain schema version 2. All 482 tests pass. ## 1.6.0 (2026-07-01) * First-class journal-capture primitives (slice 1): `PhaseCapture.stepCosts()`, `JournalSteps.fromEvents()`, production fail-loud `RunRecorder`, `JournalStorage.persistsDerivedEvents()`, per-turn usage in the immutable log * A1: `AttributionMethod.EVEN_SPLIT` — coarse fallback when per-turn tokens are unavailable * A5: per-file `{"@type":"header","schemaVersion":1}` line on `events.jsonl` and `analysis.jsonl` (loaders skip it) * Cost-metering fix: headline `LLMCallEvent.tokenUsage` is now the cost-bearing Σ-per-turn-by-type aggregate (incl. cache) instead of the last-`ResultMessage` snapshot, which under-counted \~2× on long runs — same field, corrected value, no schema change * Designed as an additive change to the capture contract; 482 tests green. No cross-version reader test was run. Ships in agentworks-bom 1.13.0 ## 1.5.0 (2026-06-17) * docs: fix TraceContentMode import path (moved to journal.trace in capture-fidelity R2) * R2.10 emission: wire StepCostEvent into analysis.jsonl from capture * Stage R2.9: Implement computeJudgeAgreement (judge\<->human calibration) * Stage R2.6: Per-step goal/outcome channel (StepOutcomeEvent) * Stage R2.10: DerivedEvent hierarchy + StepCostEvent + analysis.jsonl sidecar * Stage R2.8: First-class Gemini capture (gemini-cli-capture module) * Stage R2.7: Move portable TraceWriter into journal-core * Stage R2.5a: Delineate sub-agent spawns (interiors are out-of-stream) * Stage R2.4: Emit derived step\_cost lines into the trace (epic gate) * Stage R2.3b: Per-step cost attribution + JournalStep * Stage R2.3a: Stable step identity for events (fixes subjectId fragility) * Stage R2.2: Per-turn usage capture (TurnUsage + ModelCost from rawJson) * Stage R2.1: Consume rawJson — verbatim raw trace line (rawMode=NONE|FULL) * Stage R2.0: Bump claude-code-sdk to 1.4.0 (unblocks rawJson consumption) * CI: enforce OWASP CVE gate via reusable security-scan workflow * Add OWASP dependency-check CVE gate (-Powasp) + Trivy security-scan script * docs: version pins to 1.4.0 ## 1.4.0 (2026-06-06) * Fix CI: build on Java 21 to match claude-code-sdk artifact class version * Steps 1.1-1.2: TraceWriter v2 content capture with TraceContentMode ## 1.3.0 (2026-06-04) * Include tool inputs in TraceWriter JSONL and escape control characters ## 1.2.0 (2026-05-19) * Add feedback event type, eval subjects, and feedback service ## 1.1.0 (2026-05-15) * Update claude-code-sdk dependency to released 1.1.0 * Skip tests on snapshot publish — CI handles test validation * Update the `claude-code-sdk` dependency and imports to its `io.github.markpollack` coordinates ## 1.0.1 (2026-04-30) * Bump version to 1.0.1-SNAPSHOT * Open JournalEvent to external implementations; add registerEventType API * Update README: released version 0.9.0 * Fix release workflow: add contents write permission for git tagging ## 0.9.0 (2026-03-29) * Initial release. # API Reference Source: https://lab.pollack.ai/docs/agent-judge/api-reference Main public contracts and packages in Agent Judge 0.15 This page is a map of the 0.15 API. The generated aggregate Javadoc remains the detailed signature reference, and the [tutorial source](https://github.com/markpollack/agent-judge-tutorial) supplies compiled usage examples. ## Packages | Package | Main contents | | --------------------------------------------- | ---------------------------------------------------------------- | | `io.github.markpollack.judge` | `Judge`, `AsyncJudge`, metadata, naming, and composition | | `io.github.markpollack.judge.context` | `JudgmentContext` and `ExecutionStatus` | | `io.github.markpollack.judge.result` | `Judgment`, `JudgmentStatus`, and `Check` | | `io.github.markpollack.judge.fs` | Core file-system judges | | `io.github.markpollack.judge.jury` | Juries, verdicts, voting strategies, and policies | | `io.github.markpollack.judge.ai` | Model-backed judge composition and classifiers | | `io.github.markpollack.judge.ai.model` | Framework-neutral model requests, responses, messages, and usage | | `io.github.markpollack.judge.ai.prompt` | Prompt templates, renderers, variables, and text sources | | `io.github.markpollack.judge.exec` | Build, command, and class-version judges | | `io.github.markpollack.judge.coverage` | JaCoCo coverage judges and parser | | `io.github.markpollack.judge.file` | Semantic file judges | | `io.github.markpollack.judge.file.comparator` | Java, Maven, and XML comparators | | `io.github.markpollack.judge.llm` | Spring AI judging adapter and correctness judge | | `io.github.markpollack.judge.rag` | RAG context helpers and judges | | `io.github.markpollack.judge.springai` | Spring AI evaluated-side bridge | | `io.github.markpollack.judge.langchain4j` | LangChain4j evaluated-side bridge | | `io.github.markpollack.judge.koog` | Koog evaluated-side bridge | | `io.github.markpollack.judge.agentclient` | AgentClient evaluated-side bridge and judging adapter | ## Judge contracts `Judge` is a functional interface whose single method maps `JudgmentContext` to `Judgment`. `AsyncJudge` supplies the `CompletableFuture` variant. `JudgeWithMetadata` marks a judge that exposes `JudgeMetadata`. `DeterministicJudge` is the reusable base for rule-based implementations, and `NamedJudge` attaches identity to another judge without changing its evaluation. `Judges` contains logical composition and naming helpers. ## JudgmentContext `JudgmentContext` is the immutable in-process input record. It carries goal, workspace, execution duration, start time, optional agent output, execution status, optional error, and arbitrary context metadata. The context metadata may contain framework-native objects because it does not cross the result boundary. Bridge modules use it to expose relevant runtime evidence to ordinary judges. `ExecutionStatus` values are `SUCCESS`, `FAILED`, `TIMEOUT`, `CANCELLED`, `REFUSED`, and `UNKNOWN`. ## Judgment `Judgment` is the immutable portable output record. | Component | Contract | | ----------- | ---------------------------------------------------- | | `status` | Required `PASS`, `FAIL`, `ABSTAIN`, or `ERROR` | | `score` | Optional finite `Double` normalized to `[0.0, 1.0]` | | `label` | Optional non-blank classification | | `reasoning` | Required text and non-blank for `ABSTAIN` or `ERROR` | | `checks` | Immutable granular Boolean checks | | `metadata` | Recursively portable and immutable evidence | Direct factories are `pass(reasoning)`, `fail(reasoning)`, `abstain(reasoning)`, and `error(reasoning)`. `verdict(boolean)` starts an enrichable Boolean result without storing a duplicate score. `scored(normalizedScore).passingAt(threshold)` builds a measured result whose status follows an explicit threshold. The raw-scale overload normalizes a bounded source measurement before the threshold is applied. `effectiveScore()` returns the explicit score when present, derives `1.0` or `0.0` for an unscored `PASS` or `FAIL`, and is empty for `ABSTAIN` or `ERROR`. `elapsed()` derives `Duration` from the portable `elapsedMillis` metadata value. ## Check `Check` is a named Boolean sub-assertion with a message. Use its passing and failing factories to explain the evidence contributing to an overall result. ## Jury contracts `Jury.vote(context)` returns a `Verdict` containing aggregate and individual evidence. `SimpleJury` runs peer judges, while `CascadedJury` evaluates ordered tiers. Voting strategies are `MajorityVotingStrategy`, `ConsensusStrategy`, `AverageVotingStrategy`, `WeightedAverageStrategy`, and `MedianVotingStrategy`. `TiePolicy`, `ErrorPolicy`, and `TierPolicy` make edge behavior explicit. See [Jury System](/docs/agent-judge/jury-system) for their reduction rules. ## AI composition `ModelBackedJudge` combines `JudgePromptTemplate`, `JudgeModel`, and `JudgmentClassifier`. `JudgmentClassifiers` supplies common classifiers, and `LabelJudgmentClassifier` maps explicit labels to outcomes and optional declared scores. `JudgeModelRequest` and `JudgeModelResponse` are framework-neutral. `Usage` records optional provider-reported input, output, reasoning, cache-creation, cache-read, and total token quantities, and projects them to portable metadata. It never derives price or assumes that the categories sum to the reported total. ## Framework bridges | Evaluator | Context builder | Backend type | | ---------------------- | ----------------------------------- | ------------------------ | | `SpringAiEvaluator` | `SpringAiJudgmentContextBuilder` | Spring AI `ChatResponse` | | `LangChain4jEvaluator` | `LangChain4jJudgmentContextBuilder` | LangChain4j `Result` | | `KoogEvaluator` | `KoogJudgmentContextBuilder` | Koog `AIAgent` | | `AgentClientEvaluator` | `AgentClientJudgmentContextBuilder` | AgentClient | `SpringAiJudgeModel` and `AgentClientJudgeModel` adapt those backends to the judging-side `JudgeModel` interface. ## Version and migration Use all Agent Judge artifacts at version `0.15.0`. Consumers migrating from the former score hierarchy should follow the [normalized Judgment migration guide](https://github.com/markpollack/agent-judge/blob/main/consumer-handoff-normalized-judgment.md). # Built-in Judges Source: https://lab.pollack.ai/docs/agent-judge/built-in-judges Deterministic, execution, file, LLM, RAG, and requirements judges in Agent Judge 0.16 Every built-in judge returns the normalized `Judgment` model. Status is required, while a stored score and label are present only when that judge completed a measurement or classification. ## Core file-system judges Artifact: `agent-judge-core`. | Judge | Purpose | | ------------------- | ----------------------------------------------------------------------------------- | | `FileExistsJudge` | Check that a workspace-relative path exists | | `FileContentJudge` | Compare file content with exact, contains, or regular-expression matching | | `SupersetDiffJudge` | Require output to contain a byte-exact reference project plus any allowed additions | These judges produce Boolean outcomes and therefore normally store no score. The executable starting point is [tutorial module 01](https://github.com/markpollack/agent-judge-tutorial/tree/main/module-01-single-judge). **Changed in 0.16.0 — degenerate inputs no longer pass.** An empty path, `.`, a directory, an absolute path outside the workspace, and a parent traversal are each refused rather than treated as a satisfied check. Previously some of these returned `PASS` without examining anything. This is the failure mode judges are most prone to: **a broken judge fails open with a plausible result.** A test that cannot run reports a failure, but a judge that examines nothing still returns a status, and `PASS` on an unexamined thing is indistinguishable from `PASS` on a verified one. If you have suites that were quietly passing on a mistyped or empty path, they will start failing — that is the fix working, not a regression. ## Command and coverage judges Artifact: `agent-judge-exec`. | Judge | Purpose | | --------------------------- | ---------------------------------------------------------------------------- | | `BuildSuccessJudge` | Run Maven, Gradle, or a configured build command and judge its exit status | | `CommandJudge` | Execute an arbitrary command through Agent Sandbox and compare the exit code | | `ClassVersionJudge` | Validate Java class-file bytecode versions | | `CoveragePreservationJudge` | Reject a JaCoCo coverage regression beyond the configured tolerance | | `CoverageImprovementJudge` | Measure normalized coverage improvement with an optional floor | Build and command judges run real processes in the context workspace. Use the project wrapper when present and choose timeouts appropriate to untrusted work. [Tutorial module 02](https://github.com/markpollack/agent-judge-tutorial/tree/main/module-02-build-judge) executes the Maven build path. ## Semantic file judges Artifact: `agent-judge-file`. | Judge | Purpose | | --------------------- | ----------------------------------------------------- | | `FileComparisonJudge` | Dispatch files to the appropriate semantic comparator | | `JavaSemanticJudge` | Compare Java syntax trees rather than formatting | | `MavenSemanticJudge` | Compare important Maven model structure | | `XmlSemanticJudge` | Compare XML document structure | | `TextFileJudge` | Compare whitespace-normalized text | These judges expect reference and actual file locations in the evaluation context as documented in the aggregate Javadoc. ## LLM judge Artifact: `agent-judge-llm`. | Judge | Purpose | | ------------------ | -------------------------------------------------------------- | | `CorrectnessJudge` | Ask a Spring AI-backed model whether output satisfies the goal | | `LLMJudge` | Base template for specialized Spring AI LLM judges | For a framework-neutral composition, use `ModelBackedJudge` from `agent-judge-ai-core` with a `JudgePromptTemplate`, `JudgeModel`, and `JudgmentClassifier`. [Tutorial module 08](https://github.com/markpollack/agent-judge-tutorial/tree/main/module-08-model-backed-judge) runs that complete path with no credentials. ## RAG judges Artifact: `agent-judge-rag`. | Judge | Question answered | | -------------------------- | ----------------------------------------------------------- | | `FaithfulnessJudge` | Is the generated answer supported by the retrieved context? | | `ContextualRelevanceJudge` | Is the retrieved context relevant to the question? | | `HallucinationJudge` | Does the answer introduce unsupported claims? | `RagContext` defines the `rag.question`, `rag.context`, and `rag.answer` metadata keys and the framework fallback conventions. When required evidence is absent, these judges abstain rather than inventing a score. ## Framework evaluators Evaluators are bridges rather than judges. They execute or adapt framework output into `JudgmentContext`, then apply an ordinary `Judge` or `Jury`. | Artifact | Evaluator | Evaluated value | | -------------------------- | ---------------------- | ------------------------------------ | | `agent-judge-spring-ai` | `SpringAiEvaluator` | Spring AI `ChatResponse` | | `agent-judge-langchain4j` | `LangChain4jEvaluator` | LangChain4j `Result` | | `agent-judge-koog` | `KoogEvaluator` | Koog `AIAgent` output | | `agent-judge-agent-client` | `AgentClientEvaluator` | AgentClient responses and workspaces | The credential-free bridge samples are [Koog module 09](https://github.com/markpollack/agent-judge-tutorial/tree/main/module-09-koog-evaluation) and [LangChain4j module 10](https://github.com/markpollack/agent-judge-tutorial/tree/main/module-10-langchain4j-evaluation). ## Requirements judges New in 0.16.0. Artifact: `agent-judge-ai-core`. These run a **written requirements document back against the implementation it describes**. The input is something somebody wrote before the code existed — acceptance criteria, architectural constraints — and the question is whether the implementation satisfies it. | Type | Purpose | | ------------------- | ---------------------------------------------------------- | | `EarsCriterion` | One acceptance criterion, parsed from the specification | | `EarsJudge` | Answers every acceptance criterion | | `Rfc2119Constraint` | One architectural constraint, with its RFC 2119 keyword | | `Rfc2119Judge` | Answers every architectural constraint | | `Observation` | Non-binding evidence noticed while establishing a judgment | ### The document supplies the roster This is what separates these from the other judges, and it changes how they aggregate. Elsewhere a jury samples a population and an `ABSTAIN` is dropped as not applicable. Here the roster is fixed: the document says the requirement applies, so *"could not be established"* is not *"does not apply."* The rollup is therefore strict, and abstentions are **not** dropped: ``` any ERROR -> ERROR else any FAIL -> FAIL else any ABSTAIN -> ABSTAIN else -> PASS ``` Fifty-one criteria established and one unsettled is `ABSTAIN`, not `PASS`. The specification has not been shown to hold — which is a different statement from it having been shown to fail, and both are different from success. Understand this before composing these judges with anything else, because a strategy that drops abstentions will silently convert "unverified" into "fine." ## Related * [Getting started](/docs/agent-judge/getting-started) * [Writing custom judges](/docs/agent-judge/custom-judge) * [Jury system](/docs/agent-judge/jury-system) # Writing Custom Judges Source: https://lab.pollack.ai/docs/agent-judge/custom-judge Implement status-first deterministic and model-backed judges Choose the smallest extension point that expresses your policy. | Need | Use | | ----------------------------------------- | --------------------------- | | A local rule with no reusable identity | A `Judge` lambda | | A named lambda | `Judges.named(...)` | | A reusable rule-based class | Extend `DeterministicJudge` | | Prompt, model, and classifier composition | `ModelBackedJudge` | | A specialized Spring AI template method | Extend `LLMJudge` | ## Return a status-first result For a Boolean policy, return `Judgment.pass(reasoning)`, `Judgment.fail(reasoning)`, or begin with `Judgment.verdict(boolean)` when you also need checks or metadata. Do not store `1.0` or `0.0` merely to repeat the status. For a real measurement, begin with `Judgment.scored(normalizedScore)` and finish by declaring the passing threshold. For a categorical policy, set an optional label in addition to the required status. Use `Judgment.abstain(reasoning)` when the judge is not applicable or lacks required evidence. Use `Judgment.error(reasoning)` when evaluation could not complete, and log the originating exception where it was caught. ## Start with a lambda The complete compiled lambda examples live in [tutorial module 06](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-06-lambda-judge/src/main/java/io/github/markpollack/judge/tutorial/module06/LambdaJudgeDemo.java). They cover a three-line file rule, a named wrapper, jury participation, and error conversion. A plain lambda has no discoverable identity. Wrap it with `Judges.named` when verdicts and logs need a stable name, description, and `JudgeType`. ## Build a reusable deterministic judge Extend `DeterministicJudge` when the rule has constructor configuration, helper methods, or granular checks. Its constructor supplies `JudgeMetadata`; the subclass implements only `judge(JudgmentContext)`. [PackageStructureJudge.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-07-deterministic-judge/src/main/java/io/github/markpollack/judge/tutorial/module07/PackageStructureJudge.java) is the canonical compiled example. It resolves paths from the context workspace, records three `Check` values, derives the overall status with `Judgment.verdict`, and converts file-reading failures to a reasoned `ERROR`. ## Compose a model-backed judge `ModelBackedJudge` keeps three concerns independent. 1. `JudgePromptTemplate` renders evaluation inputs from a context. 2. `JudgeModel` invokes a backend. 3. `JudgmentClassifier` converts the response into a normalized judgment. [ModelBackedJudgeDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-08-model-backed-judge/src/main/java/io/github/markpollack/judge/tutorial/module08/ModelBackedJudgeDemo.java) compiles and runs this pipeline with a stub model. Production adapters include `SpringAiJudgeModel` and `AgentClientJudgeModel`. `LabelJudgmentClassifier` requires an explicit label-to-status policy. Assign a normalized score only when the label policy genuinely declares a measurement. Unknown labels produce a reasoned `ABSTAIN` with the raw judge output in metadata and neither a label nor a score. ## Keep metadata portable Judgment metadata may contain only portable primitives, arrays, and string-keyed objects. Convert SDK objects to stable fields before attaching them. Keep live exceptions and framework-native objects in logs or `JudgmentContext`, not in the result. ## Test all outcome paths A custom judge test should cover at least its passing, failing, abstaining, and error paths when those paths exist. Assert the status first, then assert optional score, label, checks, reasoning, and portable metadata that the policy intentionally produces. ## Related * [Executable tutorial](/docs/agent-judge/tutorial) * [Result API](/docs/agent-judge/api-reference#judgment) * [Jury system](/docs/agent-judge/jury-system) # Design Philosophy Source: https://lab.pollack.ai/docs/agent-judge/design-philosophy Status-first results, portable evidence, small contracts, and cost-aware composition ## Judges are executable acceptance criteria A judge plays the role that an assertion plays in ordinary software. It receives the evidence from an agent run and produces an explicit outcome with reasoning. Agent Judge keeps that contract small: `Judge` is a functional interface with one `judge(JudgmentContext)` method. Lambdas work directly, metadata can be added through composition, and richer implementations can extend `DeterministicJudge` or `LLMJudge`. ## Status is the required outcome A `Judgment` records three independent facts. | Fact | Presence | Meaning | | -------- | -------- | -------------------------------------------------- | | `status` | Required | `PASS`, `FAIL`, `ABSTAIN`, or `ERROR` | | `score` | Optional | A completed measurement normalized to `[0.0, 1.0]` | | `label` | Optional | The exact category assigned by a classifier | A Boolean decision stores its status without also storing `1.0` or `0.0`. This avoids two fields representing the same fact and later disagreeing. `effectiveScore()` derives a numeric view when a voting strategy needs one. The constructor enforces the cross-field rules. `ABSTAIN` and `ERROR` cannot carry a score, `ERROR` cannot carry a label, and both require non-blank reasoning. ## Errors are outcomes, not exception transport `ERROR` means a judge did not complete its evaluation. The judgment carries human-readable reasoning but no `Throwable`. The original exception should be logged where it was caught. This keeps the result deterministic and portable while preserving framework-native diagnostics in the in-process `JudgmentContext` when a judge needs them. ## Result metadata crosses boundaries Result metadata accepts only values that preserve their meaning across JSON and process boundaries. Supported values are strings, booleans, interoperable integers, finite numbers, arrays, and string-keyed objects, recursively. The result constructor normalizes, copies, and freezes this graph. SDK response objects, exceptions, enums, arbitrary-precision values, and Java time objects are rejected rather than failing later during serialization. Timing demonstrates the pattern. The portable result stores `elapsedMillis`, and `Judgment.elapsed()` derives a Java `Duration` view. ## Framework-neutral does not mean dependency-free The core API has no dependency on an agent runtime such as Spring AI, LangChain4j, Koog, or AgentClient. It does use Jackson, SLF4J, and JSpecify to implement serialization, logging, and declared nullness. Framework-specific bridges sit in separate modules. Applications add only the bridge and judge families that they need. ## Composition separates policy from evidence Judges produce individual evidence. Voting strategies decide how that evidence becomes a jury verdict. Status-counting policies such as majority and consensus reason directly about outcomes. Numeric policies such as weighted average and median use each eligible judgment's `effectiveScore()`. Abstentions and errors are not silently converted into zero measurements; the selected strategy and error policy decide how they participate. ## Cascades make cost explicit A `CascadedJury` evaluates tiers in order. Cheap deterministic or structural checks can reject invalid work before an LLM-backed tier incurs latency and token use. | Tier | Typical work | Relative cost | | ------------- | -------------------------------- | ------------------------ | | Deterministic | File existence and content rules | Microseconds | | Structural | Java, Maven, and XML comparison | Milliseconds | | Execution | Builds, tests, and coverage | Seconds to minutes | | Semantic | LLM and RAG evaluation | Model latency and tokens | The final tier is explicit, and each earlier tier declares whether it can reject or accept without escalation. ## Examples remain executable The separate [Agent Judge Tutorial](https://github.com/markpollack/agent-judge-tutorial) owns canonical examples. Its ten Maven modules compile and run without credentials, so documentation can link to maintained source instead of carrying divergent sample programs. ## Related * [Result and package reference](/docs/agent-judge/api-reference) * [Jury system](/docs/agent-judge/jury-system) * [Writing a custom judge](/docs/agent-judge/custom-judge) * [0.14 migration guide](https://github.com/markpollack/agent-judge/blob/main/consumer-handoff-normalized-judgment.md) # Getting Started with Agent Judge Source: https://lab.pollack.ai/docs/agent-judge/getting-started Add status-first evaluation to a Java 21 agent pipeline Agent Judge evaluates an agent execution after it runs. A `Judge` receives a `JudgmentContext` and returns a `Judgment` whose required status is independent from its optional normalized score and optional label. ## Prerequisites * Java 21 or newer * Maven 3.9 or the Maven Wrapper * Credentials only when you choose a live LLM-backed judge The canonical tutorial modules use deterministic inputs and require no credentials. ## Add the dependency ```xml theme={null} io.github.markpollack agent-judge-core 0.15.0 ``` All ten artifacts use the same group and version. | Artifact | Add it for | | -------------------------- | ------------------------------------------------------------- | | `agent-judge-core` | Core contracts, status-first results, file checks, and juries | | `agent-judge-ai-core` | Framework-neutral model-backed judges | | `agent-judge-exec` | Command, build, and coverage checks | | `agent-judge-file` | Semantic file comparison | | `agent-judge-llm` | Spring AI as a judging backend | | `agent-judge-rag` | RAG quality judges | | `agent-judge-spring-ai` | Evaluate Spring AI output | | `agent-judge-langchain4j` | Evaluate LangChain4j output | | `agent-judge-koog` | Evaluate Koog output | | `agent-judge-agent-client` | Evaluate or judge through AgentClient | The framework bridge dependencies are optional at the application boundary. Your application supplies the framework runtime version it uses. ## Build a context and run one judge This excerpt is compiled as [tutorial module 01](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-01-single-judge/src/main/java/io/github/markpollack/judge/tutorial/module01/SingleJudgeDemo.java). ```java theme={null} Path workspace = Path.of("test-workspace"); JudgmentContext context = JudgmentContext.builder() .goal("Add a HelloController class") .workspace(workspace) .status(ExecutionStatus.SUCCESS) .startedAt(Instant.now()) .executionTime(Duration.ofSeconds(5)) .build(); Judge fileJudge = new FileExistsJudge( "src/main/java/com/example/HelloController.java"); Judgment result = fileJudge.judge(context); ``` Read `result.status()` first. `result.score()` and `result.label()` may be `null` because a Boolean decision does not duplicate its outcome as a stored number or category. Use `result.effectiveScore()` only when an aggregation algorithm deliberately needs a numeric view of `PASS` or `FAIL`. ## Understand result portability A result can carry strings, booleans, interoperable integers, finite numbers, arrays, and string-keyed objects in metadata. Agent Judge recursively copies and freezes those values. Live exceptions, SDK responses, `Duration` objects, enums, and other Java identities do not belong in result metadata. `JudgmentContext` is an in-process input and may hold framework-native objects needed by a judge. `Judgment` is the portable output boundary. ## Run the tutorial Clone `https://github.com/markpollack/agent-judge-tutorial.git`, enter the checkout, run `./mvnw clean test`, and then run `./mvnw exec:java -pl module-01-single-judge`. The tutorial contains ten modules and progresses through composition, juries, custom judges, model-backed evaluation, Koog, and LangChain4j. ## Next Run every compiled sample Learn the normalized result invariants Choose a judge family Aggregate multiple judgments # Jury System Source: https://lab.pollack.ai/docs/agent-judge/jury-system Aggregate status-first judgments with explicit voting and error policies A `Jury` applies multiple judges and returns a `Verdict`. The verdict retains the aggregated judgment, ordered individual judgments, named evidence, weights, and any nested tier verdicts. ## SimpleJury `SimpleJury` runs peer judges, optionally in parallel, and delegates aggregation to a `VotingStrategy`. Its builder accepts named or anonymous judges, per-judge weights, a strategy, a parallel flag, and an optional executor. The canonical compiled construction and verdict inspection live in [tutorial module 04](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-04-simple-jury/src/main/java/io/github/markpollack/judge/tutorial/module04/SimpleJuryDemo.java). Read `verdict.aggregated().status()` for the outcome. Use `individualByName()` and `individual()` for diagnostics, and use the aggregate metadata's `aggregation` object for portable policy evidence. ## Voting strategies | Strategy | Reduction | | ------------------------- | ----------------------------------------------------------------- | | `MajorityVotingStrategy` | Counts applicable `PASS` and `FAIL` statuses | | `ConsensusStrategy` | Requires every applicable judgment to agree | | `AverageVotingStrategy` | Averages eligible `effectiveScore()` values | | `WeightedAverageStrategy` | Computes a weighted average of eligible `effectiveScore()` values | | `MedianVotingStrategy` | Uses the median eligible `effectiveScore()` | | `ConjunctiveStrategy` | Reduces eligible `effectiveScore()` values with `min` | | `AllMustPassStrategy` | Requires every applicable judgment to be `PASS` | Status-counting strategies do not consult stored scores. Numeric strategies use an explicit score when present and derive `1.0` or `0.0` from `PASS` or `FAIL` otherwise. `ABSTAIN` and `ERROR` never become numeric zero by default because they are not completed measurements. ### Compensatory and non-compensatory aggregation The distinction that decides which of these you want, and the one a default quietly makes for you. `AverageVotingStrategy` is **compensatory**: a high score on one criterion offsets a low score on another. That is right when the criteria trade off against each other — a slower solution that is markedly clearer may genuinely be the better one. `ConjunctiveStrategy` is **non-compensatory**: it reduces with `min`, so a single low assessment decides the aggregate and nothing else can lift it. That is right when the criteria are independently necessary. Correctness is not offset by elegance; a migration that loses data is not redeemed by good naming. `AllMustPassStrategy` is the same conjunction taken over **outcomes** rather than scores. Every applicable judgment must return `PASS`, and a mixed jury is a rejection. Use it as a gate, where the question is admissibility rather than quality. Averaging a jury whose criteria are independently necessary is the most common way a rubric reports a healthy number for work that is unusable. If any single criterion failing should sink the result, an average will not say so — it will report the mean and look reasonable doing it. ## Tie and error policies `TiePolicy` selects `PASS`, `FAIL`, or `ABSTAIN` when a strategy has no winner. The default tie policy for majority voting is `FAIL`. `ErrorPolicy` makes error handling observable. | Policy | Effect | | ------------------ | ---------------------------------------------------------------- | | `PROPAGATE` | Return an aggregate `ERROR` immediately; this is the default | | `TREAT_AS_FAIL` | Include the errored judgment as a failure | | `TREAT_AS_ABSTAIN` | Convert it to a non-vote while recording that conversion | | `IGNORE` | Remove it from the aggregation population and weight denominator | The original error judgment remains in the verdict's individual evidence even when a policy excludes it from reduction. ## Consensus semantics Consensus returns `PASS` when all applicable judges pass and `FAIL` when all applicable judges fail. A mixed applicable panel returns `ABSTAIN` because it has no collective finding; a downstream gate or tier decides whether disagreement rejects or escalates. If no applicable judges remain after abstention and error handling, the result is also `ABSTAIN`, with reasoning and aggregation evidence distinguishing the cases. ## CascadedJury `CascadedJury` orders juries into named tiers. Each tier has one control-flow policy. | Tier policy | Behavior | | -------------------- | ------------------------------------------------- | | `REJECT_ON_ANY_FAIL` | Stop and reject if any judge in the tier fails | | `ACCEPT_ON_ALL_PASS` | Stop and accept if every judge in the tier passes | | `FINAL_TIER` | Always produce the final verdict and must be last | This separates cost control from a judge's evaluation logic. A typical cascade runs deterministic checks, then structural checks, then an LLM-backed final tier only when earlier evidence cannot decide. [Tutorial module 05](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-05-cascaded-jury/src/main/java/io/github/markpollack/judge/tutorial/module05/CascadedJuryDemo.java) is the canonical compiled two-tier example. ## Identity and composition Wrap a lambda with `Judges.named` so verdicts have stable identities. The `Juries` utility builds a jury from judges and combines existing juries into a meta-jury. `Verdict.single(name, judgment)` creates complete evidence for a one-member jury without duplicating builder calls. ## Choosing a policy | Situation | Suggested policy | | --------------------------------------- | ---------------- | | Equal peer decisions | Majority | | Every requirement must hold | Consensus | | Measurements have different importance | Weighted average | | Outlier-resistant numeric reduction | Median | | Cheap gates before expensive evaluation | Cascaded jury | ## Related * [Executable tutorial](/docs/agent-judge/tutorial) * [Built-in judges](/docs/agent-judge/built-in-judges) * [API reference](/docs/agent-judge/api-reference) # Research Foundations Source: https://lab.pollack.ai/docs/agent-judge/research-foundations Why Agent Judge favors explicit requirements, executable evidence, staged evaluation, and independent checks Agent Judge evaluates whether work satisfies an explicit definition of done, using executable evidence and independent checks—not whether it resembles one reference answer. ## The motivating failure The investigation began when exact reference-file comparison rated two independently successful Spring Boot migrations as failures. Both implementations built, but each made valid choices that differed from the reference, including the framework version it targeted. The comparison answered *“Does this look like the reference?”* when the useful question was *“Does this satisfy the migration goal?”* Exact comparison is still appropriate when exact output is part of the specification. For open-ended generation, refactoring, and migration, it should be one possible check rather than the complete definition of correctness. ## Five principles | Principle | What it means | Agent Judge mechanism | | ----------------------------------- | -------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | | Evidence before opinion | Prefer builds, tests, runtime behavior, bytecode, and coverage when those facts can answer the question directly. | Command, build, class-version, and coverage judges | | Judge requirements, not resemblance | Express the invariants the result must satisfy instead of assuming one reference implementation is uniquely correct. | Composable deterministic, file-semantic, execution, RAG, and model-backed judges | | Escalate in stages | Run cheap, decisive checks before slower or more subjective evaluation. | `CascadedJury` with explicit tier policies | | Compose independent perspectives | Combine checks with different failure modes and make their aggregation policy visible. | `SimpleJury`, `MetaJury`, majority, consensus, average, weighted-average, and median strategies | | Preserve uncertainty | Keep disagreement, lack of applicability, and evaluator failure distinct from a negative finding. | `PASS`, `FAIL`, `ABSTAIN`, and `ERROR` outcomes with structured aggregation evidence | These are design principles, not claims that Agent Judge implements every technique proposed in the literature. Calibrated guarantees of human agreement, dynamic judge-team selection, multi-round debate, and formal program-equivalence proofs remain research or application-level concerns. ## Ground semantic judgment in tools Research on tool-augmented LLM evaluation found that web search and code execution can improve judge performance in many, though not all, settings. Code-evaluation systems likewise use execution and other tools to supplement model judgment. The practical implication is bounded: a semantic judge can add value, but it should not replace a build, test, or runtime check when one is available. Agent Judge therefore treats deterministic and execution-based checks as first-class judges. Model-backed judges are intended for criteria-based questions that the available tools cannot settle directly. ## Allow more than one correct implementation Program-equivalence research shows that structural difference is not evidence of behavioral difference, while current models still struggle with difficult equivalence cases. Migration benchmarks and production migration systems evaluate combinations of compilation, tests, dependency state, structural invariants, and runtime behavior rather than relying on textual similarity alone. This leads to a requirements-based evaluation model: specify what must remain true, what must change, and the evidence that demonstrates both. ## Use cascades and juries deliberately Selective-evaluation research motivates starting with lower-cost evaluators and escalating uncertain cases. Ensemble-judge research highlights the value—and limitations—of combining perspectives. Agent Judge exposes those choices as application policy: which judges participate, how their findings aggregate, how errors are handled, and when another tier runs. A jury is therefore a composition of visible judgments, not merely several models producing one opaque score. ## Protect the definition of done Migration research also documents why weak success criteria are dangerous. An agent can appear to improve a metric by removing tests, reducing exercised behavior, or satisfying only a superficial similarity check. Coverage preservation, compilation, test execution, and project-specific structural invariants make such shortcuts observable. Agent Judge supplies mechanisms for these safeguards. The application still owns its definition of done; no generic judge suite can infer all of a project's requirements automatically. ## Source map ### Evaluation architectures * [Findeis et al., *Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?*](https://aclanthology.org/2025.acl-long.779/) * [Jung et al., *Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreement*](https://arxiv.org/abs/2407.18370) * [Li et al., *LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods*](https://arxiv.org/abs/2412.05579) * [Wang et al., *CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation*](https://arxiv.org/abs/2504.13472) * [Zhou et al., *An LLM-as-Judge Metric for Bridging the Gap with Human Evaluation in SE Tasks*](https://arxiv.org/abs/2505.20854) * [Yu, *When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMs*](https://arxiv.org/abs/2508.02994) * [He et al., *LLM-as-a-Judge for Software Engineering: Literature Review, Vision, and the Road Ahead*](https://arxiv.org/abs/2510.24367) ### Equivalence and verification * [Wei et al., *EquiBench: Benchmarking Code Reasoning Capabilities of Large Language Models via Equivalence Checking*](https://arxiv.org/abs/2502.12466) * [Dilhara et al., *Unprecedented Code Change Automation: The Fusion of LLMs and Transformation by Example*](https://doi.org/10.1145/3643755) ### Migration evaluation * [Ziftci et al., *Migrating Code At Scale With LLMs At Google*](https://arxiv.org/abs/2504.09691) * [Liu et al., *MIGRATION-BENCH: Repository-Level Code Migration Benchmark from Java 8*](https://arxiv.org/abs/2505.09569) * [May et al., *FreshBrew: A Benchmark for Evaluating AI Agents on Java Code Migration*](https://arxiv.org/abs/2510.04852) * [Amin et al., *JMigBench: A Benchmark for Evaluating LLMs on Source Code Migration*](https://arxiv.org/abs/2602.09930) * [Cheng et al., *CODEMENV: Benchmarking Large Language Models on Code Migration*](https://arxiv.org/abs/2506.00894) Results and numerical claims in these sources belong to their respective studies. They are research inputs to Agent Judge's design, not Agent Judge benchmark results. ## Related * [Design philosophy](/docs/agent-judge/design-philosophy) * [Built-in judges](/docs/agent-judge/built-in-judges) * [Jury system](/docs/agent-judge/jury-system) * [Executable tutorial](/docs/agent-judge/tutorial) # Executable Tutorial Source: https://lab.pollack.ai/docs/agent-judge/tutorial Ten credential-free Maven modules for Agent Judge 0.15 The [Agent Judge Tutorial repository](https://github.com/markpollack/agent-judge-tutorial) is the canonical sample suite. Each module is a small executable Maven project compiled against Agent Judge 0.15. Clone the repository, run `./mvnw clean test`, and then run a module with `./mvnw exec:java -pl module-01-single-judge`. No module needs an API key or a live model endpoint. ## Learning path | Module | Focus | Canonical source | | ------ | ---------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 01 | Build a `JudgmentContext` and apply one `FileExistsJudge` | [SingleJudgeDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-01-single-judge/src/main/java/io/github/markpollack/judge/tutorial/module01/SingleJudgeDemo.java) | | 02 | Run a real Maven build through `BuildSuccessJudge` | [BuildJudgeDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-02-build-judge/src/main/java/io/github/markpollack/judge/tutorial/module02/BuildJudgeDemo.java) | | 03 | Compose judges with `Judges.and`, `or`, and `negate` | [CompositionDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-03-composition/src/main/java/io/github/markpollack/judge/tutorial/module03/CompositionDemo.java) | | 04 | Aggregate named judges with majority and weighted-average policies | [SimpleJuryDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-04-simple-jury/src/main/java/io/github/markpollack/judge/tutorial/module04/SimpleJuryDemo.java) | | 05 | Arrange cost-aware tiers with `CascadedJury` | [CascadedJuryDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-05-cascaded-jury/src/main/java/io/github/markpollack/judge/tutorial/module05/CascadedJuryDemo.java) | | 06 | Write lambda judges and attach discoverable metadata | [LambdaJudgeDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-06-lambda-judge/src/main/java/io/github/markpollack/judge/tutorial/module06/LambdaJudgeDemo.java) | | 07 | Implement a reusable `DeterministicJudge` with granular checks | [PackageStructureJudge.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-07-deterministic-judge/src/main/java/io/github/markpollack/judge/tutorial/module07/PackageStructureJudge.java) | | 08 | Compose a prompt, model adapter, and classifier with `ModelBackedJudge` | [ModelBackedJudgeDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-08-model-backed-judge/src/main/java/io/github/markpollack/judge/tutorial/module08/ModelBackedJudgeDemo.java) | | 09 | Evaluate deterministic Koog agent output through `KoogEvaluator` | [KoogEvaluationDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-09-koog-evaluation/src/main/java/io/github/markpollack/judge/tutorial/module09/KoogEvaluationDemo.java) | | 10 | Evaluate a deterministic LangChain4j `Result` through `LangChain4jEvaluator` | [LangChain4jEvaluationDemo.java](https://github.com/markpollack/agent-judge-tutorial/blob/main/module-10-langchain4j-evaluation/src/main/java/io/github/markpollack/judge/tutorial/module10/LangChain4jEvaluationDemo.java) | ## What to observe Modules 01 through 07 establish the framework-neutral contract. Read the required status first and treat a stored score or label as optional evidence. Module 08 demonstrates that AI-backed evaluation is composition rather than inheritance. Its stub model makes the complete prompt-to-classifier path executable without credentials. Modules 09 and 10 replace the former README-only bridge samples. They execute real adapter code with deterministic framework objects, so bridge construction and metadata extraction remain verifiable offline. ## Framework versions used by the samples | Integration | Tutorial target | | ----------- | --------------- | | Agent Judge | `0.15.0` | | Koog | `1.1.1` | | LangChain4j | `1.19.0` | Spring AI and AgentClient integration APIs are covered by the library's bridge modules and tests. The tutorial's framework examples deliberately avoid online calls and credentials. ## Continue Choose an aggregation policy Turn domain rules into judgments # What's New Source: https://lab.pollack.ai/docs/agent-judge/whats-new Curated Agent Judge release notes ## 0.15.0 Agent Judge 0.15 is a compatibility-preserving dependency-alignment and release-reproducibility update. It keeps the 0.14 status-first `Judgment` API, the Java 21 baseline, and all ten published modules unchanged. ### Consumer dependency alignment * Standalone consumers continue to resolve Jackson 2.21.6, with Jackson Annotations on its 2.21 release line. * `agent-judge-llm` and `agent-judge-rag` now declare Jackson 3.1.6 in their flattened POMs so consumers without the AgentWorks BOM receive the accepted Jackson 3 floor. * The provided Agent Client integration baseline advances to 0.26.0. Applications still choose and provide their Agent Client runtime. ### Agent Sandbox Core `agent-judge-exec` now uses `agent-sandbox-core` 0.10.0. That Core release protects environment values in `ExecSpec.toString()` and avoids forwarding the whole parent process environment when caller overrides are present. Agent Judge's own execution paths construct specs with a command and timeout only; they do not set environment variables. This release does not add the Sandbox Docker or E2B modules. ### Reproducible release evidence * Build and snapshot workflows are pinned to the reviewed Build Tools commit, and the Build workflow supports exact-SHA manual verification. * Every module publishes binary, source, and generated Javadoc JARs with the project-specific Business Source License terms. * Maven Central attaches one aggregate CycloneDX 1.6 SBOM to the parent coordinate. It is supporting inventory for the reactor and included dependency graph; provided-scope inventory is not a claim that every listed component enters an application's runtime closure. See the complete [0.15 release notes](https://github.com/markpollack/agent-judge/blob/v0.15.0/RELEASE_NOTES_0.15.md) and the [GitHub release](https://github.com/markpollack/agent-judge/releases/tag/v0.15.0). ## 0.14.0 Agent Judge 0.14 replaces the former sealed score hierarchy with a normalized, portable `Judgment` contract. This is a breaking pre-1.0 release, so consumers must recompile and follow the [migration guide](https://github.com/markpollack/agent-judge/blob/main/consumer-handoff-normalized-judgment.md). ### Result model * Every judgment has a required `PASS`, `FAIL`, `ABSTAIN`, or `ERROR` status. * A normalized score and a classification label are independent optional facts. * Boolean outcomes do not duplicate status as a stored `1.0` or `0.0`; `effectiveScore()` supplies an intentional numeric view for aggregation. * Errors carry reasoned outcomes rather than live exceptions. ### Portable evidence * Result metadata is validated as portable primitives, arrays, and string-keyed objects, then recursively copied and frozen. * Invalid values identify their exact metadata path. * Timing is stored as integer `elapsedMillis`, with `Judgment.elapsed()` providing the Java `Duration` view. * Model usage records separately reported input, output, reasoning, cache-creation, cache-read, and total token quantities without deriving price. ### Aggregation * Every voting strategy exposes an `ErrorPolicy`, defaulting to `PROPAGATE`. * Aggregates contain structured population and error-accounting evidence. * Consensus returns `ABSTAIN` when applicable judges disagree, leaving rejection or escalation to a downstream gate or tier. ### API and dependencies * The `io.github.markpollack.judge.score` package and `ReactiveJudge` are removed. * The core remains agent-runtime-neutral and declares its real Jackson Databind, SLF4J API, and JSpecify dependencies. * JSpecify exposes optional score and label nullness in the Java API, with NullAway enforcing the adopted packages. ### Framework baselines The release candidate is verified against the final stable framework matrix: | Integration | Version | | ------------ | ------------------------------------------------------------- | | Spring AI | `2.0.0` with Spring Boot `4.1.0` and Spring Framework `7.0.8` | | LangChain4j | `1.19.0` | | Koog | `1.1.1` | | Agent Client | `0.25.0` | Framework bridge dependencies remain `provided` where practical so applications select their runtime. ### Samples and reference * The [Agent Judge Tutorial](https://github.com/markpollack/agent-judge-tutorial) now owns ten canonical credential-free Maven modules. * Executable Koog and LangChain4j modules replace the former README-only bridge samples. * Aggregate Javadoc is warning-clean, warnings are release-breaking, and every published module attaches a Javadoc JAR. * See the complete [0.14 release notes](https://github.com/markpollack/agent-judge/blob/main/RELEASE_NOTES_0.14.md). ### Release packaging * The current source tree uses the project-specific Business Source License terms in the repository's [root `LICENSE`](https://github.com/markpollack/agent-judge/blob/main/LICENSE). Earlier published releases keep the licenses shipped with those releases. * Maven Central publication attaches one CycloneDX 1.6 aggregate SBOM to the parent coordinate, covering the ten reactor modules and their included external dependencies. * The exact-pinned release workflow runs the full test-bearing verification before deploy and uses the curated `RELEASE_NOTES_0.14.md` body. * The GitHub Release is notes and source archives only; the signed Maven artifacts and aggregate SBOM belong on Maven Central, with no separately uploaded GitHub release assets. ## 0.13.0 * Upgraded to Spring AI 2.0.0 GA and its supported Spring Boot 4.1.0 baseline. * Added an OWASP dependency-check gate. ## 0.12.0 * Aligned Jackson dependencies for the release line. ## 0.11.0 * Added `agent-judge-ai-core` and framework-neutral model-backed judge composition. ## 0.10.0 * Added Spring AI, LangChain4j, Koog, and AgentClient bridges. * Added RAG evaluation judges and the nine-module architecture that preceded `agent-judge-ai-core`. ## 0.9.x * Moved public coordinates to `io.github.markpollack`. * Added superset project comparison and the initial public release. # What's New Source: https://lab.pollack.ai/docs/agent-memory/whats-new Curated release notes for Agent Memory ## 0.4.0 (2026-08-18) **First Business Source License 1.1 release, with supply-chain and disclosure work. No behavioural change to Tier-1 compaction.** ### License * Current development and 0.4.0 and later are licensed under **Business Source License 1.1**. **0.3.0 and earlier remain Apache License 2.0**, and published tags and artifacts are not retroactively relicensed. The historical Apache text is retained in the repository as `LICENSE-APACHE.txt`. * Both license files are staged into every published binary, source, and Javadoc archive. ### Dependency resolution * Jackson 2 raised to 2.21.6 and Jackson 3 to 3.1.6. * Jackson 3 core and databind are now declared **directly** on `memory-core`, so a standalone consumer with no AgentWorks BOM and no Jackson management of its own resolves 3.1.6 by Maven nearest-wins. Parent BOM imports align the build reactor but do not travel with a published module to a downstream consumer — that gap is what this change closes. * A repository-owned standalone-consumer resolution gate is committed and runs in CI after the clean build. * The unused Spring milestone and snapshot resolution repositories are removed; Spring AI 2.0.0 GA resolves from Maven Central. ### Supply chain * The parent artifact now publishes one aggregate **CycloneDX 1.6 JSON SBOM** (`classifier=cyclonedx`) covering both modules and their shipped dependency closure. * **Hosted OWASP/NVD dependency scanning in GitHub Actions has been removed.** Vulnerability analysis is now a local offline Trivy procedure run against a validated database snapshot. There is no hosted CVE gate in this project's CI. * Reusable build workflows are pinned to a build-tools commit SHA, and the external actions this repository references directly are pinned to full-length commit SHAs. The pinned reusable workflows still resolve some actions internally by moving reference, so the complete executed CI path is not yet immutable. ### Documented operating boundary The filesystem store is **local, plaintext, and single-writer**. There is no locking, no atomic index replacement, and no crash recovery: a second concurrent writer, or a crash during an index write, can corrupt or truncate `_index.json`. Stored memory is injected verbatim into the model prompt, so only trusted content should be written to it. Multi-process or concurrent-writer use requires external coordination. These are disclosed limits, not shipped features. ### Unchanged `FileSystemMemoryStore`, `ProgressFileMemoryStore`, `MemoryCompactor` and `CompactionMemoryAdvisor` behaviour, the public API, and the on-disk format are unchanged. This release is not a Jackson 3 source migration. ## 0.3.0 (2026-06-15) * Upgraded to Spring AI 2.0.0 GA. * Added a hosted OWASP dependency-check CVE gate. **This gate was removed again in 0.4.0** in favour of a local offline procedure. ## 0.2.0 (2026-06-06) * Aligned on the Jackson 2.21.2 BOM. No functional change to compaction. ## 0.1.0 (2026-04-02) * Initial release. Tier-1 compaction: `MemoryStore`, `FileSystemMemoryStore`, `MemoryCompactor`, `TokenEstimator`, and the `CompactionMemoryAdvisor` Spring AI `BaseAdvisor`. # What's New Source: https://lab.pollack.ai/docs/agent-sandbox/whats-new Release notes for Agent Sandbox ## 0.10.0 (2026-08-18) * Restore the convenient local Docker backend on current Docker Engine versions with Testcontainers 1.21.4, and add its core-and-Docker TCK run to ordinary CI. * Require callers to select the Docker image. The no-argument `DockerSandbox()` constructor is removed, `builder().build()` fails before Docker access unless `.image(...)` is set, and explicit-image constructors continue to work. Agent Sandbox does not ship or maintain a runtime image; the caller owns the selected image's security and update policy. * Preserve argument-vector semantics across backends and harden Docker environment and file-list execution against shell injection. * Stop copying the complete host environment into `LocalSandbox`, update Jackson and commons-compress, package license texts in distributed archives, and attach a graph-derived CycloneDX 1.6 SBOM to the parent artifact. Three disclosed upstream findings remain in Apache HttpComponents classes embedded and relocated inside the Java Docker transport. The HTTP/2/HPACK finding is not reachable on the Docker HTTP/1.1 path examined. The other two require a malicious response from the trusted, root-equivalent local Docker daemon; no path from untrusted code inside the selected container to those advisory conditions was identified under this trust model. Published docker-java/Testcontainers combinations examined do not contain fixed embedded versions, and ordinary dependency management cannot override classes already relocated inside the zerodep JAR. The findings remain visible and are accepted for the trusted-local-daemon use case. Changing the selected container image cannot remove them. The CycloneDX SBOM is derived from the Maven dependency graph and does not enumerate components shaded into dependency JARs. For this release, its scan must be read alongside the actual-JAR scan that detects the three transport findings; a zero-finding SBOM scan is a coverage limitation, not a clean closure. ## 0.9.3 (2026-06-06) * Align Jackson to 2.21.2 via jackson-bom import; release 0.9.3 ## 0.9.2 (2026-05-15) * Migrate to markpollack org: package rename, BSL license, standalone POM, build-tools workflows * Add README with Maven Central coordinates and docs link * Fix release workflow: add contents write permission for git tagging ## 0.9.1 (2026-03-05) * Initial release. # AgentLoop Source: https://lab.pollack.ai/docs/agent-workflow/agent-loop Ready-to-use SWE agent — tools, session memory, observability, and a simple run/chat API backed by Spring AI's agent loop ## What is AgentLoop? `AgentLoop` is a ready-to-use agent that packages everything you need to run an autonomous coding task: tools, system prompt, turn limits, cost limits, session memory, and observability. Under the hood it delegates to Spring AI's `ChatClient` + `AgentLoopAdvisor` for the actual tool-calling loop. Think of it as "batteries included" — one class, one builder call, working agent: ```java theme={null} AgentLoop agent = AgentLoop.builder() .config(AgentLoop.Config.builder() .maxTurns(10) .workingDirectory(Path.of("/my/project")) .build()) .model(chatModel) .build(); AgentLoop.Result result = agent.run("Add unit tests for the UserService class"); System.out.println(result.status()); // COMPLETED System.out.println(result.output()); // "I've added 5 tests..." System.out.println(result.totalTokens()); // 12400 ``` ## Built-in Tools AgentLoop ships with a full set of SWE tools: | Tool | Source | Description | | -------------------------------- | --------------------- | ---------------------------------------- | | `Bash` | workflow-tools | Shell commands (git, mvn, docker) | | `Read` / `Write` / `Edit` / `LS` | spring-ai-agent-utils | File operations | | `Glob` | spring-ai-agent-utils | Find files by pattern | | `Grep` | spring-ai-agent-utils | Search file contents | | `TodoWrite` | spring-ai-agent-utils | Task planning and tracking | | `Task` | spring-ai-agent-utils | Delegate to sub-agents | | `Submit` | AgentLoop (internal) | Submit final answer, terminates the loop | ## Configuration `AgentLoop.Config` is a Java record with a builder: ```java theme={null} AgentLoop.Config config = AgentLoop.Config.builder() .maxTurns(20) // default: 20 .costLimit(1.0) // default: $1.00 .commandTimeout(Duration.ofSeconds(30)) // default: 30s .workingDirectory(Path.of(".")) // default: user.dir .systemPrompt("Custom prompt...") // default: built-in SWE prompt .build(); ``` | Parameter | Default | Description | | ------------------ | ------------------- | -------------------------------------- | | `maxTurns` | 20 | Max LLM invocations before termination | | `costLimit` | 1.0 | Max estimated cost in USD | | `commandTimeout` | 30s | Timeout for shell commands | | `workingDirectory` | `user.dir` | Root directory for file/shell tools | | `systemPrompt` | Built-in SWE prompt | System instructions for the agent | Configs are immutable records. Use `toBuilder()` to derive variants: ```java theme={null} AgentLoop.Config extended = config.toBuilder() .maxTurns(50) .build(); ``` ## Session Memory Enable session memory to preserve conversation context across multiple calls: ```java theme={null} AgentLoop agent = AgentLoop.builder() .config(config) .model(chatModel) .sessionMemory() // default in-memory implementation .build(); agent.run("Read the UserService class and summarize it"); agent.run("Now add tests for the methods you found"); // remembers context agent.clearSession(); // reset when done ``` You can also pass a custom `ChatMemory` implementation: ```java theme={null} .sessionMemory(myJdbcChatMemory) ``` ## Interactive Mode For TUI/CLI applications, interactive mode enables human-in-the-loop via `AskUserQuestionTool`: ```java theme={null} AgentCallback callback = new AgentCallback() { @Override public void onThinking() { System.out.print("Thinking..."); } @Override public String onQuestion(List questions) { // Present questions to user, return their answer return scanner.nextLine(); } @Override public void onComplete() { System.out.println("Done."); } }; AgentLoop agent = AgentLoop.builder() .config(config) .model(chatModel) .interactive(true) .agentCallback(callback) .sessionMemory() .build(); agent.chat("Review this PR and ask me about anything unclear", callback); ``` ## Result `AgentLoop.Result` is a record with execution details: ```java theme={null} AgentLoop.Result result = agent.run(task); result.status(); // COMPLETED, TURN_LIMIT_REACHED, TIMEOUT, STUCK, etc. result.output(); // Agent's final text output result.turnsCompleted(); // Number of LLM invocations result.toolCallsExecuted();// Total tool calls across all turns result.totalTokens(); // Total tokens consumed result.estimatedCost(); // Estimated cost in USD result.isSuccess(); // true if COMPLETED result.isFailure(); // true if FAILED ``` ## Observability AgentLoop wires Micrometer observations for tool call tracking. You can also provide a custom `ToolCallListener`: ```java theme={null} AgentLoop agent = AgentLoop.builder() .config(config) .model(chatModel) .toolCallListener(new LoggingToolCallListener()) .build(); ``` ## Relationship to LoopPattern `AgentLoop` is **not** a `LoopPattern` implementation. It's a higher-level construct: * **`LoopPattern`** — the abstract interface for loop algorithms (`TurnLimitedLoop`, `EvaluatorOptimizerLoop`, `StateMachineLoop`). Lives in `workflow-api`. * **`AgentLoop`** — a concrete, runnable agent that uses `AgentLoopAdvisor` internally. Lives in `workflow-agents`. `AgentLoop` is what you use when you want to *run* an agent. `LoopPattern` is what you implement when you want to *create a new loop algorithm*. ## Maven Coordinates ```xml theme={null} io.github.markpollack workflow-agents 0.10.0 ``` # Annotation Model Source: https://lab.pollack.ai/docs/agent-workflow/annotation-model Declare agents with @Agent, compose workflows inside AgentHandler, and wire exception handling with @ExceptionHandler and @AgentAdvice ## Why an Annotation Model? Steps and workflows are the plumbing. Agents are the product — the thing you expose to HTTP endpoints, MCP servers, message listeners, and other agents. The annotation model gives you a programming model for declaring agents, naming them, describing them, handling their errors, and looking them up by name at runtime. If you've used `@Controller` + `@ControllerAdvice` in Spring MVC, you already know how this works. `@Agent` is to `AgentHandler` what `@Controller` is to your handler methods. ## AgentHandler\ The entry-point contract for agents. A `@FunctionalInterface` that takes context and input, returns output: ```java theme={null} @FunctionalInterface public interface AgentHandler { O handle(AgentContext ctx, I input); } ``` **Lambda form** — for simple cases: ```java theme={null} AgentHandler echo = (ctx, input) -> input.toUpperCase(); String result = echo.handle(AgentContext.create(), "hello"); // "HELLO" ``` **Class-based form** — for production agents that compose a `Workflow` internally: ```java theme={null} public class CodeReviewAgent implements AgentHandler { private final Step fetchDiff = Step.named("fetch-diff", (ctx, prUrl) -> gitHub.fetchDiff(prUrl)); private final Step analyze = Step.named("analyze", (ctx, diff) -> llm.analyze(diff)); @Override public ReviewReport handle(AgentContext ctx, String prUrl) { return Workflow.define("code-review") .step(fetchDiff) .then(analyze) .run(prUrl, ctx); } } ``` `AgentHandler` is pure Java — no Spring dependency. It lives in `workflow-api` so it's portable across any runtime. ## @Agent Marks a class as a named agent and registers it as a Spring bean: ```java theme={null} @Agent("code-review") public class CodeReviewAgent implements AgentHandler { // ... } ``` `@Agent` is a Spring `@Component` stereotype — classes annotated with it are picked up by component scanning. The `value()` is the unique agent name used for registry lookup and protocol addressing. ## @StepName and @Description Two metadata annotations for steps and agents: ```java theme={null} @StepName("analyze-diff") @Description("Parses a PR diff and extracts changed files and hunks") public class AnalyzeDiffStep implements Step { // ... } ``` **@StepName** pins a stable name for checkpoint and Temporal keys. When you rename the class, the checkpoint key stays the same — crash recovery and Temporal replay still match the right execution record. Without it, `Step.name()` (which defaults to the class simple name) is used. **@Description** provides a human-readable description for traces, visualization, and protocol tool listings. Can be placed on types or methods. Both are pure Java annotations — no Spring dependency, no runtime cost. ### StepNames utility `StepNames.resolve(step)` reads the `@StepName` annotation; falls back to `step.name()` if absent: ```java theme={null} StepNames.resolve(new AnalyzeDiffStep()); // "analyze-diff" (from annotation) StepNames.resolve(Step.named("foo", ...)); // "foo" (from name() method) ``` ## AgentRegistry An immutable map of name → handler. Manual construction — you decide what goes in: ```java theme={null} AgentRegistry registry = new AgentRegistry(Map.of( "code-review", codeReviewAgent, "security-audit", securityAuditAgent )); AgentHandler agent = registry.get("code-review").orElseThrow(); Set names = registry.names(); // ["code-review", "security-audit"] ``` The registry is defensive-copied at construction (`Map.copyOf()`). Subsequent changes to the source map don't affect it. Auto-configuration that scans `@Agent` beans and populates `AgentRegistry` automatically is planned but not yet implemented. For now, construct the registry manually or in a `@Configuration` class. ## Exception Handling Exception handling mirrors Spring MVC's `@ExceptionHandler` + `@ControllerAdvice` pattern exactly. Two scopes: 1. **Per-agent** — `@ExceptionHandler` methods inside an `@Agent` class handle that agent's exceptions 2. **Cross-cutting** — `@ExceptionHandler` methods inside an `@AgentAdvice` class handle exceptions from any agent Per-agent handlers take priority. When multiple handlers match, the most specific exception type wins (subclass beats superclass). ### Per-agent handling ```java theme={null} @Agent("code-review") public class CodeReviewAgent implements AgentHandler { @Override public ReviewReport handle(AgentContext ctx, String prUrl) { return Workflow.define("code-review") .step(fetchDiff) .then(analyze) .run(prUrl, ctx); } @ExceptionHandler(RateLimitException.class) public ReviewReport handleRateLimit(RateLimitException ex) { return new ReviewReport("Rate limited — try again later", "SKIP"); } @ExceptionHandler(IllegalStateException.class) public ReviewReport handleBudget(IllegalStateException ex, AgentContext ctx) { log.warn("Agent {} exceeded budget", ctx.runId()); return new ReviewReport("Budget exceeded: " + ex.getMessage(), "SKIP"); } } ``` Handler methods support two signatures: * `ReturnType method(ExceptionType ex)` — just the exception * `ReturnType method(ExceptionType ex, AgentContext ctx)` — exception plus context The exception type can be declared explicitly in `@ExceptionHandler(SomeException.class)` or inferred from the method's first parameter. ### Cross-cutting handling ```java theme={null} @AgentAdvice public class GlobalErrorHandler { @ExceptionHandler(Exception.class) public Object handleAny(Exception ex, AgentContext ctx) { log.error("Agent {} failed: {}", ctx.runId(), ex.getMessage()); return Map.of("error", ex.getMessage(), "runId", ctx.runId()); } } ``` `@AgentAdvice` is a Spring `@Component` — picked up by component scanning, applies to all agents. ### Using the resolver `AgentExceptionHandlerResolver` ties it together. Pass it the `@AgentAdvice` instances at construction, then call `resolve()` on exception: ```java theme={null} var resolver = new AgentExceptionHandlerResolver(List.of(globalErrorHandler)); try { return agent.handle(ctx, input); } catch (Exception ex) { return resolver.resolve(agent, ex, ctx) .orElseThrow(() -> ex); // re-throw if no handler matched } ``` Resolution order: per-agent handlers on the agent instance first, then cross-cutting advice in registration order. ## Spring Boot Wiring In a Spring Boot application, wire the registry and resolver as beans in a `@Configuration` class: ```java theme={null} @Configuration(proxyBeanMethods = false) public class AgentConfig { @Bean AgentExceptionHandlerResolver agentExceptionHandlerResolver( MyErrorAdvice errorAdvice) { // @AgentAdvice bean return new AgentExceptionHandlerResolver(List.of(errorAdvice)); } @Bean AgentRegistry agentRegistry(PrReviewAgent prReviewAgent) { return new AgentRegistry(Map.of("pr-review", prReviewAgent)); } } ``` Spring injects the `@Agent` and `@AgentAdvice` beans by type. The registry and resolver become injectable anywhere you need them — HTTP controllers, message listeners, MCP tool handlers. ### Disambiguating steps with @Qualifier When two steps share the same `Step` signature, Spring can't tell them apart by type alone. Use `@Qualifier` on both the bean definition and the injection point: ```java theme={null} @Component @Qualifier("assess-code-quality") @StepName("assess-code-quality") public class AssessCodeQualityStep implements Step { // ... } @Component @Qualifier("assess-backport") @StepName("assess-backport") public class AssessBackportStep implements Step { // ... } ``` Then inject by qualifier in the agent constructor: ```java theme={null} @Agent("pr-review") public class PrReviewAgent implements AgentHandler { public PrReviewAgent( @Qualifier("assess-code-quality") Step quality, @Qualifier("assess-backport") Step backport) { // ... } } ``` This is the standard Spring pattern — `@StepName` gives you a stable checkpoint key, `@Qualifier` gives you injection disambiguation. ## Injecting Durability with withExecutor() By default, `Workflow.run()` creates a `WorkflowExecutor` with `LocalStepRunner` (in-process, no persistence). Use `withExecutor()` to inject a configured executor with a durable `StepRunner`: ```java theme={null} // Configure executor with JDBC checkpointing WorkflowExecutor executor = new WorkflowExecutor( checkpointingStepRunner, jdbcTraceRecorder ); // Inject into workflow — both run() and build() paths work String result = Workflow.define("durable-review") .withExecutor(executor) .step(fetchDiff) .then(analyze) .run(prUrl, ctx); // Or build a reusable Workflow that remembers the executor Workflow workflow = Workflow.define("review") .withExecutor(executor) .step(fetchDiff) .then(analyze) .build(); workflow.execute(ctx, prUrl); // uses the injected executor ``` This works on both `WorkflowBuilder` and `SupervisorBuilder`. See [Durability](/docs/agent-workflow/durability) for the full graduation path from `LocalStepRunner` → `CheckpointingStepRunner` → `TemporalStepRunner`. ## Full Example A complete PR review agent using all 5 annotations: ```java theme={null} // -- Domain types -- record PrEvent(String prUrl, String diff) {} record ReviewReport(String summary, String recommendation) {} // -- Steps with stable names -- @StepName("analyze-diff") @Description("Parses a PR diff and extracts changed files") public class AnalyzeDiffStep implements Step { @Override public String execute(AgentContext ctx, PrEvent event) { return llm.analyze(event.diff()); } @Override public String name() { return "AnalyzeDiffStep"; } } @StepName("produce-review") @Description("Produces a structured review report") public class ProduceReviewStep implements Step { @Override public ReviewReport execute(AgentContext ctx, String analysis) { return new ReviewReport(analysis, "LGTM"); } @Override public String name() { return "ProduceReviewStep"; } } // -- Agent -- @Agent("pr-review") @Description("Reviews a pull request and produces a structured report") public class PrReviewAgent implements AgentHandler { private final Step analyze = new AnalyzeDiffStep(); private final Step review = new ProduceReviewStep(); @Override public ReviewReport handle(AgentContext ctx, PrEvent event) { return Workflow.define("pr-review") .step(analyze) .then(review) .run(event, ctx); } @ExceptionHandler(IllegalStateException.class) public ReviewReport handleBudget(IllegalStateException ex) { return new ReviewReport("Budget exceeded: " + ex.getMessage(), "SKIP"); } } // -- Cross-cutting error handling -- @AgentAdvice public class GlobalErrorHandler { @ExceptionHandler(Exception.class) public Object handleAny(Exception ex, AgentContext ctx) { return new ReviewReport("Error: " + ex.getMessage(), "ERROR"); } } // -- Wiring -- AgentRegistry registry = new AgentRegistry(Map.of( "pr-review", new PrReviewAgent() )); var resolver = new AgentExceptionHandlerResolver( List.of(new GlobalErrorHandler()) ); ``` A runnable version of this example — with real LLM calls, `AgentRegistry` lookup, `StepNames` resolution, and exception handler tests — is in [AnnotationModelIT.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/src/test/java/io/github/markpollack/workflow/examples/ours/AnnotationModelIT.java) in the [workflow-dsl-examples](https://github.com/markpollack/workflow-dsl-examples) repo. ## Annotation Summary | Annotation | Target | Module | Purpose | | ------------------- | ------------ | -------------- | -------------------------------------------------- | | `@StepName` | Type | workflow-api | Stable checkpoint/Temporal key | | `@Description` | Type, Method | workflow-api | Human-readable description | | `@ExceptionHandler` | Method | workflow-api | Marks exception handler methods | | `@Agent` | Type | workflow-flows | Spring stereotype for `AgentHandler` beans | | `@AgentAdvice` | Type | workflow-flows | Spring stereotype for cross-cutting error handlers | The first three are pure Java — zero Spring dependency, portable across any runtime. The last two are Spring stereotypes that enable component scanning and auto-discovery. ## What's Next Crash recovery with CheckpointingStepRunner and Temporal integration Branch, loop, parallel, decision, gate, supervisor — 10+ composable patterns Step, AgentContext, Gate, WorkflowGraph, StepRunner Steps, context, your first workflow # Agent Workflow API Reference Source: https://lab.pollack.ai/docs/agent-workflow/api-reference Step interface, Workflow builder, WorkflowGraph IR, gates, context, StepRunner, and TraceRecorder ## Step\ The atomic unit of a workflow. ```java theme={null} public interface Step { String name(); O execute(AgentContext ctx, I input); default AgentContext updateContext(AgentContext ctx, O output) { return ctx; } default Class inputType() { return Object.class; } default Class outputType() { return Object.class; } static Step named(String name, BiFunction fn); static Step noop(); // anonymous pass-through static Step noop(String name); // named pass-through — referenceable in IR } ``` **`updateContext()`** — override to publish side-channel metadata alongside the primary output. Called by the executor after `execute()`. The `output` parameter is the value returned by `execute()` — derive metadata from it directly rather than capturing state in fields: ```java theme={null} class ClassifierStep implements Step { @Override public ClassificationResult execute(AgentContext ctx, String input) { return runModel(input); // rich result carries all data } @Override public AgentContext updateContext(AgentContext ctx, ClassificationResult output) { return ctx.mutate() .with(CONFIDENCE, output.confidence()) .with(DETECTED_LABEL, output.label()) .build(); } } ``` The primary output (`ClassificationResult`) flows forward as the next step's input; metadata (confidence, label) flows via context keys. Default implementation returns `ctx` unchanged. **AgentStep** marker — implement on steps that make LLM calls for correct `NodeType.AGENT` cost tracking: ```java theme={null} public interface AgentStep { /* marker — no methods */ } ``` ## Built-in Step Types | Type | Factory | What it wraps | | ---------- | -------------------------------------------------- | ---------------------------------------- | | Lambda | `Step.named("n", (ctx, in) -> ...)` | Any function | | Chat | `ChatClientStep.of(chat, "template {input}")` | Single Spring AI call | | Claude | `ClaudeStep.of("template").workingDirectory(path)` | Full Claude CLI session | | A2A | `A2AStep.of(url)` | Remote agent via Agent-to-Agent protocol | | Function | `Steps.of(fn)` | Pure deterministic function | | Retry | `Steps.retrying(3, step)` | Retry wrapper | | Output ref | `Steps.outputOf("step-name")` | Read prior step result from context | | Terminate | `Steps.terminate(status, msg)` | Early workflow exit | ### ClaudeStep options ```java theme={null} ClaudeStep.of("Fix the tests in {input}") .workingDirectory(projectPath) .permissionMode(PermissionMode.ACCEPT_EDITS) .withMcp(mcpConfig) .withA2a(a2aEndpoint); ``` ## Workflow Builder ```java theme={null} Workflow.define("name") .step(step) // first step .then(step) // chain .branch(predicate).then(a).otherwise(b) // conditional .gather(step1, step2) // homogeneous fan-out → List .parallel(step1, step2) // enrichment fan-out → fork input unchanged, branches write to context .parallel(itemsSupplier, stepFactory) // dynamic fan-out .repeatUntil(predicate).step(a).end() // while-do .repeatUntilOutput(predicate).step(a).end() // do-while .decision(chat).option("a", step).end() // LLM routing .gate(gate).onPass(a).onFail(b).maxRetries(3).end() // quality gate .onError(ExType.class, recovery) // error recovery .run(input) // execute .run(input, RunOptions.maxCost(5.0)) // with constraints .compile() // → WorkflowGraph (IR only) .build() // → Workflow (implements Step) ``` Named shortcuts: ```java theme={null} Workflow.sequential("name").step(a).then(b).run(input); Workflow.loop("name").step(a).times(5).run(input); Workflow.loop("name").step(a).until(predicate).run(input); Workflow.parallel("name").step(a, b, c).run(input); Workflow.supervisor("name", chat).agents(a, b).until(pred).run(input); ``` ## Gate ```java theme={null} public interface Gate { GateDecision evaluate(AgentContext ctx, O output); } ``` `GateDecision`: `PASS`, `FAIL`, `ESCALATE`, `TIMEOUT` ### Implementations | Gate | Description | | ----------------------------- | ------------------------------------------------------ | | `JudgeGate(jury, threshold)` | Automated — passes when agent-judge score >= threshold | | `TieredGate(jury, high, low)` | >= high → PASS, >= low → ESCALATE, \< low → FAIL | | `HumanGate` | HITL — waits for external signal with durable timeout | ### Gate builder ```java theme={null} .gate(new JudgeGate(jury, 0.8)) .onPass(approveStep) // required .onFail(reviseStep) // optional .onTimeout(fallbackStep) // optional .withReflector(reflectorStep) // Verdict → feedback text for retry .maxRetries(3) // cap .end() ``` On failure, `Verdict` written to `AgentContext.JUDGE_VERDICT`. Reflector transforms verdict into constructive feedback for the retry step. ### Gate.updateContext() Gates implement `updateContext()` just like steps do. Override it to write the gate's assessment to `AgentContext` — both the `onPass` and `onFail` branches can then read it: ```java theme={null} class AssessCodeQualityGate implements Gate { static final ContextKey QUALITY_ASSESSMENT = ContextKey.of("quality-assessment", Assessment.class); private Assessment assessment; @Override public GateDecision evaluate(AgentContext ctx, PrContext pr) { assessment = runJudge(pr); return assessment.score() >= threshold ? GateDecision.PASS : GateDecision.FAIL; } @Override public AgentContext updateContext(AgentContext ctx, PrContext pr) { return ctx.mutate().with(QUALITY_ASSESSMENT, assessment).build(); } } ``` ## AgentContext Immutable, threaded through every step. Copy-on-write via `mutate()`. ### Well-known Keys | Key | Type | Description | | -------------------- | --------- | ----------------------- | | `WORKFLOW_RUN_ID` | `String` | Unique execution ID | | `WORKFLOW_NAME` | `String` | Workflow name | | `CURRENT_STEP` | `String` | Active step name | | `ITERATION_COUNT` | `Integer` | Loop counter | | `ACCUMULATED_COST` | `Double` | USD spend so far | | `ACCUMULATED_TOKENS` | `Long` | Total tokens | | `JUDGE_VERDICT` | `Verdict` | Gate failure verdict | | `JUDGE_REFLECTION` | `String` | Reflector feedback text | ```java theme={null} // Read ctx.get(AgentContext.ITERATION_COUNT) // Optional ctx.require(AgentContext.ITERATION_COUNT) // throws if missing // Write (returns new context) ctx.mutate().with(key, value).build() ``` `ContextKey` implements Bloch's Typesafe Heterogeneous Container (Effective Java Item 33). ### mergeFrom ```java theme={null} AgentContext merged = base.mergeFrom(donor); ``` Overlays all entries from `donor` onto `base`. Donor wins on key conflict. Returns a new immutable instance. Used internally by `WorkflowExecutor` to propagate sub-workflow context writes back to the parent — you rarely call this directly, but it is part of the public API if you need manual context composition. ## Sub-workflow Composition A `Workflow` implements `Step`, so it can be used anywhere a step is expected — as a sequential step, inside a branch, inside a gate path, or inside a parallel branch. The executor handles the nesting transparently: ```java theme={null} Workflow aiAssessment = Workflow.define("ai-assessment") .step(assessCodeQuality) // updateContext() writes QUALITY_ASSESSMENT .then(assessBackport) // updateContext() writes BACKPORT_ASSESSMENT .build(); Workflow.define("pr-review") .step(fetchPrContext) // writes PR_CONTEXT to ctx .branch(skipAi) .then(skipStep) .otherwise(aiAssessment) // sub-workflow — context writes propagate back .then(qualityJudge) // reads QUALITY_ASSESSMENT and BACKPORT_ASSESSMENT ✓ .run(prNumber); ``` **How it works**: The executor detects `step instanceof Workflow` and runs it inline rather than dispatching through the `StepRunner`. The sub-workflow receives the parent's current context, executes normally, and the final context (including all `updateContext()` writes from every nested step) is merged back into the parent via `mergeFrom()`. Leaf steps still go through the `StepRunner` (so `TemporalStepRunner` dispatch works correctly for individual steps). **Nesting is unlimited**: sub-workflows can contain sub-workflows. Each level merges its context writes back up the chain. **Parallel branches**: context writes from parallel sub-workflow branches are merged at the join node. If two branches write to the same key, the last branch to complete wins. ## RunOptions ```java theme={null} RunOptions.maxCost(5.0) RunOptions.maxIterations(50) RunOptions.maxDuration(Duration.ofMinutes(10)) RunOptions.unlimited() // Chain RunOptions.maxCost(5.0).withMaxIterations(50).withMaxDuration(Duration.ofMinutes(10)) ``` ## WorkflowGraph (IR) Pure data structure — no execution logic, no Spring AI imports. ```java theme={null} WorkflowGraph graph = workflow.compile(); graph.name() // String graph.nodes() // List graph.edges() // List graph.startNode() // String graph.finishNode() // String ``` ### Node Types (sealed) | Node | Description | | --------------- | ----------------------------------------------- | | `StepNode` | Regular step execution (AGENT or DETERMINISTIC) | | `GatewayNode` | Predicate branch | | `DecisionNode` | LLM routing | | `GateNode` | Quality gate with reflector and retry | | `LoopEntryNode` | While-do entry | | `LoopCheckNode` | Do-while condition | | `LoopExitNode` | Loop resume point | | `ForkNode` | Parallel split | | `JoinNode` | Convergence | ### Edge Conditions (sealed) | Condition | From | | ------------------------------- | ----------------------- | | `Unconditional` | Sequential | | `BooleanGuard(true/false)` | GatewayNode | | `OptionMatch("name")` | DecisionNode | | `GateMatch(PASS/FAIL/ESCALATE)` | GateNode | | `BranchIndex(i)` | ForkNode | | `ErrorMatch(ExType)` | Error edges | | `LoopContinue` / `LoopExit` | Loop nodes | | `BackEdge(condition)` | `backTo()` cyclic edges | ## Type Checking `WorkflowGraphAssert.assertTypeCompatible(graph)` walks a compiled graph and checks that each step's declared output type is assignable to the next step's declared input type. Catches `ClassCastException`-style bugs at test time. **Opt-in**: Steps that override `inputType()` / `outputType()` participate. Lambda steps and `Step.named()` return `Object.class` by default and are silently skipped — no false positives. ```java theme={null} // Typed steps declare their types static class ClassifyStep implements Step { @Override public Class inputType() { return String.class; } @Override public Class outputType() { return ClassificationResult.class; } // ... } static class FormatResultStep implements Step { @Override public Class inputType() { return ClassificationResult.class; } @Override public Class outputType() { return String.class; } // ... } // Compile and check — no LLM calls needed WorkflowGraph graph = Workflow.define("pipeline") .step(new ClassifyStep(chat)) .then(new FormatResultStep()) .compile(); WorkflowGraphAssert.assertTypeCompatible(graph); // passes ``` If types don't match: ```java theme={null} .step(new ClassifyStep(chat)) // outputs ClassificationResult .then(new IntegerConsumer()) // expects Integer — MISMATCH // Throws: // TypeIncompatibleException: step 'classify' outputs ClassificationResult // but step 'int-consumer' expects Integer ``` Untyped steps (lambdas) break the typed chain — no check across the gap: ```java theme={null} .step(new ClassifyStep(chat)) // typed .then(Step.named("transform", (ctx, in) -> in.toString())) // untyped — skipped .then(new IntegerConsumer()) // no check — prior step untyped ``` Use in CI to validate workflow structure without making LLM calls. ## WorkflowAbortException Thrown by a step to abort the workflow and return a typed result. Unlike an unhandled exception (which propagates as an error), `WorkflowAbortException` is caught by the executor and its carried result becomes the workflow's output: ```java theme={null} class QualityGateStep implements Step { @Override public Path execute(AgentContext ctx, ReviewResult result) { if (result.isTooRisky()) { // abort — caller receives the error report, not an exception throw new WorkflowAbortException(writeErrorReport(result)); } return writeApprovalReport(result); } } ``` Use `WorkflowAbortException` when a step detects an unrecoverable condition but can still produce a meaningful output — for example, a typed error response, a failure report, or a structured rejection. Prefer it over returning `null` or throwing a raw exception when the caller needs to distinguish "completed with degraded output" from "crashed." ## StepRunner The substrate swap seam. Same workflow code, different durability: ```java theme={null} @Bean StepRunner stepRunner() { return new LocalStepRunner(); // in-process, zero overhead } ``` Three runners are available: | Runner | What it adds | Status | | ------------------------- | ----------------------------------------------------- | --------- | | `LocalStepRunner` | Direct in-process execution, zero overhead | Available | | `CheckpointingStepRunner` | JDBC crash recovery via `workflow-batch` | Available | | `TemporalStepRunner` | Distributed durable execution via `workflow-temporal` | Available | Same workflow code — swap the `@Bean`, not the workflow. See [Durability](/docs/agent-workflow/durability) for setup and crash-recovery examples. **Operator retry (`CheckpointManager`)**: FAILED steps are not retried automatically — the system holds them until an operator decides the failure was transient. `CheckpointManager.resetFailedSteps(runId)` deletes FAILED records; re-running with the same `runId` then skips COMPLETED steps and retries the reset ones. ```java theme={null} CheckpointManager manager = new CheckpointManager(readRepo, writeRepo); manager.getRunState("run-1"); // inspect: what completed, what failed manager.resetFailedSteps("run-1"); // delete FAILED records (only if transient) executor.execute(graph, ctx, input); // retry — COMPLETED skipped, reset steps re-run ``` **Sub-workflows are always inline**: a `Workflow` used as a step bypasses the `StepRunner` entirely and runs in-process. Only leaf steps go through the runner. This is intentional — `TemporalStepRunner` dispatches to a separate activity worker thread and cannot carry full parent context; sub-workflows must stay in-process to propagate context correctly. ## TraceRecorder Records every step transition: ```java theme={null} record StepTransition( String workflowRunId, String workflowName, String fromStep, String toStep, Instant timestamp, Duration stepDuration, long tokensUsed, double costUsd, NodeType nodeType, String label, String tracePath // absolute path to JSONL trace file, or null ) {} ``` `TraceRecorder.noop()` is the default — zero overhead unless opted in. Trace data feeds Markov analysis, run diagnosis, and replay. When a step produces a trace file (via `AgentClientStep` backed by a trace-aware client), the `tracePath` is included in the transition. See [Trace Capture](/docs/agent-workflow/trace-capture) for the full guide. ## Module Structure ``` agent-workflow/ ├── workflow-api/ # Step, AgentContext, ContextKey, Gate, RunOptions ├── workflow-core/ # WorkflowGraph, WorkflowExecutor, StepRunner, TraceRecorder ├── workflow-flows/ # Workflow DSL, ClaudeStep, ChatClientStep, AgentClientStep, A2AStep, JudgeGate ├── workflow-batch/ # CheckpointingStepRunner, JdbcTraceRecorder, JPA entities ├── workflow-temporal/ # TemporalStepRunner, StepActivityImpl ├── workflow-journal/ # JournalTraceRecorder, WorkflowStepEvent ├── workflow-tools/ # BashTool, ReadTool, WriteTool, EditTool, GlobTool, GrepTool ├── workflow-agents/ # AgentLoop └── workflow-examples/ # Example pipelines ``` # DSL Primitives Source: https://lab.pollack.ai/docs/agent-workflow/choosing-a-pattern 10+ composable primitives for building agentic pipelines — sequential, branch, loop, parallel, decision, gate, supervisor, and more ## Two Entry Points ```java theme={null} // Named shortcuts — intent obvious from the first word Workflow.sequential("pipeline").step(a).then(b).then(c).run(input); Workflow.loop("refine").step(refineStep).times(5).run(draft); Workflow.parallel("gather").step(review, audit, ciCheck).run(event); // Full DSL — for gates, branches, decisions, error paths Workflow.define("complex-flow") .step(fetch) .then(analyze) .gate(judgeGate).onPass(approve).onFail(revise).end() .run(event); // Supervisor — LLM autonomously delegates to sub-agents Workflow.supervisor("delegate", routingClient) .agents(codeReview, securityAudit, docUpdate) .until(result -> result.score() >= 0.9) .run(event); ``` Both surfaces produce a `WorkflowGraph` and support `.run(input)`. ## The Primitives ### Sequential Chain steps — output flows forward: ```java theme={null} Workflow.define("pipeline") .step(write) .then(editForAudience) .then(editForStyle) .run(input); ``` ### Branch (predicate routing) Route based on output: ```java theme={null} Workflow.define("router") .step(classify) .branch(output -> "medical".equals(output)) .then(medicalExpert) .otherwise(legalExpert) .run("I broke my leg"); ``` ### Loop (while-do) Exit condition checked before body: ```java theme={null} Workflow.loop("refine") .step(refineStep) .until(result -> result.score() >= 0.8) .run(draft); ``` ### Loop (do-while / repeatUntilOutput) Body runs first, exit condition reads actual output: ```java theme={null} Workflow.define("score-loop") .repeatUntilOutput(score -> score instanceof Double d && d >= 0.8) .step(editor) .step(scorer) .end() .run(initialInput); ``` ### Gather (homogeneous fan-out) Run steps concurrently, collect all results into a `List` that becomes the input to the next step. Use when all branches produce the same type and downstream needs everything together: ```java theme={null} List results = (List) Workflow.define("gather") .gather(findMeals, findMovies) .run("romantic"); ``` Chain directly into a join step: ```java theme={null} String summary = (String) Workflow.define("parallel-review") .gather(reviewStep, auditStep) .then(Step.named("summarize", (ctx, in) -> { List parts = (List) in; return String.join("+", parts.stream().map(Object::toString).toList()); })) .run("input"); ``` ### Parallel (enrichment fan-out) Run steps concurrently; the join passes the **fork input through unchanged**. Each branch writes its output to a named context key (`Steps.outputOf(branchName)`). Use when branches produce different types and downstream reads from context: ```java theme={null} Step reviewStep = Step.named("review", (ctx, in) -> "review-done"); Step auditStep = Step.named("audit", (ctx, in) -> "audit-done"); Workflow.define("parallel-review") .parallel(reviewStep, auditStep) .then(Step.named("report", (ctx, in) -> { // in == fork input ("pr-123"), unchanged String reviewResult = ctx.get(Steps.outputOf("review")).orElseThrow(); String auditResult = ctx.get(Steps.outputOf("audit")).orElseThrow(); return reviewResult + " | " + auditResult; })) .run("pr-123"); ``` ### Parallel (dynamic fan-out) Fan-out determined at runtime: ```java theme={null} Workflow.define("batch") .parallel( ctx -> loadItems(), // items supplier item -> Step.named("process", (c, i) -> processItem(i)) // step factory ) .run(null); ``` ### Decision (LLM-routed) LLM picks which step to run: ```java theme={null} Workflow.define("decision-router") .decision(chat) .option("summarize", summarizeStep) .option("translate", translateStep) .end() .run("The quick brown fox..."); ``` ### Gate (quality checkpoint) Approve, reject, or retry with feedback: ```java theme={null} Workflow.define("gated-pipeline") .step(generate) .gate(new JudgeGate(jury, 0.8)) .onPass(approve) .onFail(revise) .withReflector(reflectorStep) // transforms verdict → feedback .maxRetries(3) .end() .run(input); ``` On failure, the full `Verdict` (score, reasoning, per-judge judgments) is written to `AgentContext` under `JUDGE_VERDICT` for the retry step to consume. ### Supervisor (autonomous delegation) LLM picks sub-agents each iteration: ```java theme={null} Workflow.supervisor("text-improver", chat) .agents(review, edit, format) .until(ctx -> ctx.get(AgentContext.ITERATION_COUNT).orElse(0) >= 3) .run(roughDraft); ``` ### Error Recovery Route exceptions to recovery steps: ```java theme={null} Workflow.define("resilient") .step(riskyStep) .onError(TimeoutException.class, retryWithBackoff) .then(finalStep) .run(input); ``` ### BackTo (cyclic back-edge) Jump back to an earlier step when a condition is met — a lightweight escape hatch for retry patterns: ```java theme={null} Workflow.define("rebase-loop") .step(rebase) .step(runTests) .backTo("rebase", result -> !"PASS".equals(result)) .step(merge) .run(branch); ``` `backTo` creates an `EdgeCondition.BackEdge` in the graph IR — a real edge, visible in traces. The next `.step()` chains from the same node (not the back-edge target), so `merge` runs when the predicate returns `false`. Use `RunOptions.maxIterations()` as a circuit breaker to prevent infinite loops: ```java theme={null} .run(input, RunOptions.maxIterations(10)); ``` **When to use `backTo` vs `repeatUntil`**: `repeatUntil` creates structured loop nodes (entry → body → check → exit). `backTo` is a single edge — no extra nodes, no loop wrapper. Use it when you need a quick "retry this section" without the overhead of a full loop construct. ### Terminate Exit early from anywhere: ```java theme={null} Steps.terminate(WorkflowStatus.FAILED, "Quality below threshold") ``` ## Step Types Steps are the atomic unit. Each wraps a different kind of execution: | Step Type | What It Wraps | Execution Time | | -------------------------------------- | ----------------------------- | -------------- | | `Step.named("n", lambda)` | Any lambda | Depends | | `ChatClientStep.of(chat, template)` | Single Spring AI call | Seconds | | `ClaudeStep.of(template)` | Full Claude CLI agent session | Minutes | | `AgentClientStep.of(client, template)` | AgentClient abstraction | Minutes | | `A2AStep.of(url)` | Remote A2A agent | Minutes | | `Steps.of(fn)` | Pure deterministic function | Milliseconds | | `Steps.retrying(n, step)` | Retry wrapper | Per attempt | | `Steps.outputOf("step-name")` | Read prior step's output | Instant | The key differentiator: `ClaudeStep` runs a full agentic loop internally — many turns, many tool calls, minutes of execution. The workflow sees it as one step. This is the opposite of frameworks where each LLM API call is a separate workflow activity. ## Composability Workflows implement `Step`. Nest them freely: ```java theme={null} var inner = Workflow.define("inner") .step(analyze).then(classify).build(); Workflow.define("outer") .step(inner) // workflow-as-step .then(report) .run(input); ``` ## RunOptions Runtime constraints — not DSL verbs, hints to the executor: ```java theme={null} Workflow.define("bounded") .step(expensiveStep) .run(input, RunOptions.maxCost(5.0) .withMaxIterations(50) .withMaxDuration(Duration.ofMinutes(10))); ``` ## The Graph IR Every primitive compiles to real nodes and typed edges — not opaque lambdas: * A **branch** is 4 nodes (gateway → then-step, else-step → join) + 4 edges * A **loop** has a back-edge from body to entry * A **parallel** has fork and join nodes with `BranchIndex` edges This makes workflows inspectable, traceable, and replayable. The `TraceRecorder` sees every transition; Markov analysis works on the edge data. ```java theme={null} WorkflowGraph graph = Workflow.define("my-flow") .step(a).then(b).compile(); graph.nodes() // List — sealed interface graph.edges() // List — typed conditions ``` ## Related Install, first workflow, four-layer architecture 8 runnable integration tests validated against GPT-4.1 # Durability Source: https://lab.pollack.ai/docs/agent-workflow/durability Crash recovery, checkpointing, and distributed execution for agent workflows ## The Graduation Path Agent Workflow separates workflow definition from execution. The `StepRunner` interface is the seam — swap the bean, not the workflow: | Level | Runner | What it adds | | ----- | ------------------------- | ----------------------------------------------------- | | 0 | `LocalStepRunner` | In-process, zero overhead. Default. | | 1 | `CheckpointingStepRunner` | JDBC crash recovery — resume from last completed step | | 2 | `TemporalStepRunner` | Distributed durable execution via Temporal activities | Same workflow code at every level: ```java theme={null} // Level 0 — default, no persistence @Bean StepRunner stepRunner() { return new LocalStepRunner(); } // Level 1 — JDBC crash recovery @Bean StepRunner stepRunner(AgentStepExecutionReadRepository readRepo, AgentStepExecutionWriteRepository writeRepo) { return new CheckpointingStepRunner(readRepo, writeRepo); } // Level 2 — Temporal durable execution @Bean StepRunner stepRunner() { return new TemporalStepRunner("agent-tasks"); } ``` ## CheckpointingStepRunner Persists step outputs to a JDBC database. On restart with the same `runId`, completed steps are skipped — their cached output is returned directly. ### How it works 1. Before executing a step, queries by `(runId, stepName)` — the **checkpoint key** 2. If a `COMPLETED` record exists, returns the cached `outputPayload` (skip) 3. Otherwise, creates a `STARTED` record, executes the step, upgrades to `COMPLETED` with the serialized output 4. On exception, records `FAILED` with the error message ### Maven coordinates ```xml theme={null} io.github.markpollack workflow-batch 0.10.0 ``` Requires Spring Data JPA and a JDBC `DataSource` on the classpath. H2 works for development; Postgres or MySQL for production. ### Restart semantics `runId` is the stable identity for a workflow instance. **COMPLETED steps are skipped permanently for that `runId`.** FAILED steps are not automatically retried — the system leaves them in place until an operator explicitly decides to retry. This is intentional. Not all failures are transient: a bad prompt, a schema mismatch, or a programming error will fail again without a fix. Automatic retry would mask the real problem. ### Crash-and-resume with CheckpointManager When a step fails, call `CheckpointManager.getRunState()` to inspect what happened, then `resetFailedSteps()` only after confirming the failure was transient: ```java theme={null} var manager = new CheckpointManager(readRepo, writeRepo); var ctx = AgentContext.withRunId("run-1"); var executor = new WorkflowExecutor(checkpointRunner, TraceRecorder.noop()); // First attempt — crashes at step-c try { executor.execute(workflow.graph(), ctx, "start"); } catch (RuntimeException e) { // step-a and step-b: COMPLETED; step-c: FAILED } // Operator inspects state List state = manager.getRunState("run-1"); // diagnose: was this a transient network blip? a permanent config error? // Only reset if the failure was transient int reset = manager.resetFailedSteps("run-1"); // deletes the FAILED record(s) // reset == 1 (step-c) // Retry with same runId — step-a and step-b skipped, step-c retried String result = executor.execute(workflow.graph(), ctx, "start"); ``` `resetFailedSteps` deletes FAILED records; COMPLETED records are untouched. The next execution creates a fresh STARTED record for each reset step and re-runs it. ### Basic crash-and-resume example A 4-step workflow crashes at step 3. After operator reset, steps 1-2 are skipped (cached), step 3 retried: ```java theme={null} Step step1 = Step.named("step-1", (ctx, in) -> in + "→1"); Step step2 = Step.named("step-2", (ctx, in) -> in + "→2"); Step step3 = Step.named("step-3", (ctx, in) -> { if (shouldCrash()) throw new RuntimeException("crash!"); return in + "→3"; }); Step step4 = Step.named("step-4", (ctx, in) -> in + "→4"); var workflow = Workflow.define("crash-resume") .step(step1).step(step2).step(step3).step(step4) .build(); // First attempt — step-3 fails var ctx = AgentContext.withRunId("run-1"); var executor = new WorkflowExecutor(checkpointRunner, TraceRecorder.noop()); executor.execute(workflow.graph(), ctx, "start"); // throws // Operator diagnoses, decides it was transient, resets new CheckpointManager(readRepo, writeRepo).resetFailedSteps("run-1"); // Retry — steps 1-2 SKIPPED (cached), steps 3-4 execute String result = executor.execute(workflow.graph(), ctx, "start"); // result: "start→1→2→3→4" ``` A complete runnable example is in [`workflow-dsl-examples/CrashRecoveryIT`](https://github.com/markpollack/workflow-dsl-examples) — `@DataJpaTest` + H2, no LLM needed. ### JPA entities Two JPA entities back the checkpoint system: | Entity | Table | Purpose | | -------------------- | ----------------------- | ----------------------------------------------------------------------------------------------------- | | `AgentStepExecution` | `agent_step_executions` | Per-step checkpoint. Key: `(runId, stepName)` unique constraint. Tracks status, output, tokens, cost. | | `AgentFlowExecution` | `agent_flow_executions` | Per-run envelope. Tracks workflow name, steps total/completed, total cost. | Both use `BatchStatus` (severity-ordered enum) and `ExitStatus` (embeddable record with severity-based composition via `and()`). ### Typed output deserialization Each checkpoint stores the step's output type alongside its serialized payload. On restore, `CheckpointingStepRunner` uses `Class.forName(outputType)` to deserialize back to the original type rather than raw `Object`. This means that when a step is skipped and its cached output is returned to the next step, the type is preserved: ```java theme={null} // Step produces a typed result Step summarize = new SummarizeStep(chat); // First run — step executes, checkpoint stores SummaryReport.class + JSON payload // Second run (restart) — checkpoint restores SummaryReport, not Map or String ``` Steps that declare `outputType()` participate fully. `Step.named()` lambdas return `Object.class` by default — deserialization falls back to Jackson's type inference for those. ## JdbcTraceRecorder Records every step transition to a `step_transitions` table. Auto-creates the table on first use. ```java theme={null} // From DataSource TraceRecorder recorder = new JdbcTraceRecorder(dataSource); // Or from JdbcTemplate TraceRecorder recorder = new JdbcTraceRecorder(jdbcTemplate); // Wire into executor var executor = new WorkflowExecutor(stepRunner, recorder); ``` Each `StepTransition` record includes: `run_id`, `workflow_name`, `from_step`, `to_step`, `timestamp`, `duration_ms`, `tokens_used`, `cost_usd`, `node_type`, `label`, `trace_path`. The `trace_path` column stores the absolute path to the step's JSONL trace file when using a trace-aware `AgentClientStep` — see [Trace Capture](/docs/agent-workflow/trace-capture). Query traces for a run: ```java theme={null} JdbcTraceRecorder recorder = new JdbcTraceRecorder(dataSource); List trace = recorder.getTrace("run-1"); ``` ## TemporalStepRunner Dispatches each step as a Temporal Activity. Steps must be registered with `StepActivityImpl` on the worker side. ### Maven coordinates ```xml theme={null} io.github.markpollack workflow-temporal 0.10.0 ``` ### Activity dispatch ```java theme={null} // Workflow side — configure the runner StepRunner runner = new TemporalStepRunner("agent-tasks"); // Default timeouts: 10min start-to-close, 30s heartbeat // Custom timeouts StepRunner runner = new TemporalStepRunner("agent-tasks", Duration.ofMinutes(30), // start-to-close Duration.ofMinutes(1)); // heartbeat ``` ### Worker-side step registration ```java theme={null} // Register steps with the activity implementation StepActivityImpl activity = new StepActivityImpl(); activity.registerStep(step1); activity.registerStep(step2); // Wire into Temporal worker Worker worker = factory.newWorker("agent-tasks"); worker.registerActivitiesImplementations(activity); ``` Steps are resolved by name from a `ConcurrentHashMap` registry. The activity creates a fresh `AgentContext` with the `runId` for each execution. Steps dispatched via Temporal must be **idempotent** — Temporal may retry activities on timeout or failure. **Sub-workflows run inline, not as activities.** A `Workflow` used as a step inside another `Workflow` bypasses the `TemporalStepRunner` and executes in-process. Only leaf steps are dispatched as Temporal activities. This is required for correct context propagation — the activity worker receives only the `runId`, not the full parent context. ## Related StepRunner interface, TraceRecorder, WorkflowExecutor Sequential, parallel, gate, loop, branch, and more # Workflow DSL Examples Source: https://lab.pollack.ai/docs/agent-workflow/examples Complete, runnable examples — validated with real LLM calls against GPT-4.1 New to Agent Workflow? Start with the [Tutorial](/docs/agent-workflow/tutorial) for a progressive introduction. This page is a complete reference of all examples. Every example below is a real integration test from [workflow-dsl-examples](https://github.com/markpollack/workflow-dsl-examples). All pass against GPT-4.1 with temperature 0.3. See also the [Annotation Model example](/docs/agent-workflow/annotation-model#full-example) for `@Agent`, `@ExceptionHandler`, `AgentRegistry`, and more. ## Setup All examples share this `ChatClient` factory: ```java theme={null} String apiKey = System.getenv("OPENAI_API_KEY"); OpenAiApi api = OpenAiApi.builder().apiKey(apiKey).build(); OpenAiChatModel model = OpenAiChatModel.builder() .openAiApi(api) .defaultOptions(OpenAiChatOptions.builder() .model("gpt-4.1") .maxTokens(1024) .temperature(0.3) .build()) .build(); ChatClient chat = ChatClient.builder(model).build(); ``` *** ## 1. Sequential Pipeline Chain steps into a pipeline — each step's output flows into the next. ```java theme={null} Step write = Step.named("write", (ctx, in) -> chat.prompt() .user("You are a creative writer. Write a 3-sentence story about: " + in) .call().content()); Step editForAudience = Step.named("edit-audience", (ctx, in) -> chat.prompt() .user("Rewrite this story for young adults. Return only the story: " + in) .call().content()); Step editForStyle = Step.named("edit-style", (ctx, in) -> chat.prompt() .user("Rewrite this story in a humorous style. Return only the story: " + in) .call().content()); String result = (String) Workflow.define("novel-creator") .step(write) .then(editForAudience) .then(editForStyle) .run("dragons and wizards"); ``` Three LLM calls in sequence: write a story, rewrite for audience, rewrite for style. *** ## 2. Branch (Predicate Routing) Route to different steps based on a classification result. ```java theme={null} Step classify = Step.named("classify", (ctx, in) -> chat.prompt() .user("Classify this as either 'medical' or 'legal'. " + "Reply with exactly one word: " + in) .call().content().strip().toLowerCase()); Step medicalExpert = Step.named("medical", (ctx, in) -> chat.prompt() .user("You are a medical expert. Briefly advise on: " + in) .call().content()); Step legalExpert = Step.named("legal", (ctx, in) -> chat.prompt() .user("You are a legal expert. Briefly advise on: " + in) .call().content()); String result = (String) Workflow.define("category-router") .step(classify) .branch(output -> "medical".equals(output)) .then(medicalExpert) .otherwise(legalExpert) .run("I broke my leg, what should I do?"); // Medical input → routes to medicalExpert assertThat(result.toLowerCase()) .containsAnyOf("doctor", "hospital", "medical", "fracture", "treatment"); ``` The `.strip().toLowerCase()` on the classify output is important — LLMs sometimes return trailing whitespace or mixed case. *** ## 3. Loop (Repeat Until Output) Iterate until a quality threshold is met. This is the most complex primitive — LLM score parsing needs care. ```java theme={null} AtomicInteger iterations = new AtomicInteger(0); Step scorer = Step.named("scorer", (ctx, in) -> { iterations.incrementAndGet(); String response = chat.prompt() .user("Rate this text for humor on a scale of 0.0 to 1.0. " + "Reply with ONLY a decimal number, nothing else: " + in) .call().content().strip(); // Parse score — regex fallback for safety try { return Double.parseDouble(response); } catch (NumberFormatException e) { var matcher = java.util.regex.Pattern.compile("\\d+\\.\\d+").matcher(response); if (matcher.find()) { return Double.parseDouble(matcher.group()); } return 0.0; // can't parse, keep looping } }); Step editor = Step.named("editor", (ctx, in) -> chat.prompt() .user("Write a very short (2-sentence) extremely funny joke about dragons. " + "Be hilarious.") .call().content()); Object result = Workflow.define("humor-loop") .repeatUntilOutput(score -> score instanceof Double d && d >= 0.6) .step(editor) .step(scorer) .end() .run("A dragon walked into a bar."); assertThat(iterations.get()).isBetween(1, 10); assertThat((Double) result).isGreaterThanOrEqualTo(0.6); ``` **Key finding**: GPT-4.1 returns clean decimal numbers every time with the "Reply with ONLY a decimal number" prompt. The regex fallback never fires — but it's there for safety with other models. *** ## 4. Parallel (Fan-Out) Run steps concurrently, collect results into a list. ```java theme={null} Step findMeals = Step.named("find-meals", (ctx, in) -> chat.prompt() .user("Suggest 3 meals for a " + in + " evening. " + "Just list the meal names, one per line.") .call().content()); Step findMovies = Step.named("find-movies", (ctx, in) -> chat.prompt() .user("Suggest 3 movies for a " + in + " evening. " + "Just list the movie titles, one per line.") .call().content()); @SuppressWarnings("unchecked") List results = (List) Workflow.define("evening-planner") .parallel(findMeals, findMovies) .run("romantic"); // results.get(0) = meal suggestions // results.get(1) = movie suggestions assertThat(results).hasSize(2); assertThat((String) results.get(0)).isNotBlank(); assertThat((String) results.get(1)).isNotBlank(); ``` Both LLM calls execute concurrently. Results are ordered to match step order. *** ## 5. Error Recovery Route exceptions to a recovery step instead of failing the workflow. ```java theme={null} Step riskyStep = Step.named("risky", (ctx, in) -> { if (((String) in).contains("bad")) { throw new IllegalArgumentException("Bad input detected"); } return chat.prompt() .user("Process this: " + in) .call().content(); }); Step recovery = Step.named("recovery", (ctx, in) -> chat.prompt() .user("The previous step failed. " + "Generate a safe default response for: " + in) .call().content()); Step finalStep = Step.named("finalize", (ctx, in) -> "Final: " + in); String result = (String) Workflow.define("error-recovery") .step(riskyStep) .onError(IllegalArgumentException.class, recovery) .then(finalStep) .run("bad input"); assertThat(result).startsWith("Final:"); ``` The exception routes to `recovery`, whose output flows into `finalStep` as if `riskyStep` had succeeded. The workflow continues — it doesn't crash. *** ## 6. Decision (LLM-Routed) Let the LLM choose which step to execute. Unlike `branch()` (predicate-based), `decision()` gives the LLM a menu of labeled options. ```java theme={null} Step summarize = Step.named("summarize", (ctx, in) -> chat.prompt() .user("Summarize this in one sentence: " + in) .call().content()); Step translate = Step.named("translate", (ctx, in) -> chat.prompt() .user("Translate this to French: " + in) .call().content()); String result = (String) Workflow.define("decision-router") .decision(chat) .option("summarize", summarize) .option("translate", translate) .end() .run("The quick brown fox jumps over the lazy dog. " + "This is a classic English pangram used for testing."); assertThat(result).isNotBlank(); assertThat(result.split("\\s+").length).isGreaterThan(3); ``` The DSL generates a routing prompt from the option names. GPT-4.1 returns clean single-word labels — no parsing issues. *** ## 7. Gate (Quality Checkpoint) Evaluate output quality and route to pass or fail paths. ```java theme={null} AtomicReference routeTaken = new AtomicReference<>(); Gate qualityGate = (ctx, output) -> { String response = chat.prompt() .user("Rate this text for quality on a scale of 0.0 to 1.0. " + "Reply with ONLY a decimal number: " + output) .call().content().strip(); double score; try { score = Double.parseDouble(response); } catch (NumberFormatException e) { var matcher = java.util.regex.Pattern.compile("\\d+\\.\\d+").matcher(response); score = matcher.find() ? Double.parseDouble(matcher.group()) : 0.0; } return score >= 0.7 ? GateDecision.PASS : GateDecision.FAIL; }; Step generate = Step.named("generate", (ctx, in) -> chat.prompt() .user("Write a well-crafted 2-sentence story about: " + in) .call().content()); Step approve = Step.named("approve", (ctx, in) -> { routeTaken.set("pass"); return "APPROVED: " + in; }); Step reject = Step.named("reject", (ctx, in) -> { routeTaken.set("fail"); return "REJECTED: " + in; }); String result = (String) Workflow.define("gated-pipeline") .step(generate) .gate(qualityGate) .onPass(approve) .onFail(reject) .end() .run("a heroic knight"); assertThat(routeTaken.get()).isIn("pass", "fail"); assertThat(result).satisfiesAnyOf( r -> assertThat(r).startsWith("APPROVED:"), r -> assertThat(r).startsWith("REJECTED:")); ``` GPT-4.1 typically produces quality text, so this usually routes to APPROVED. The gate becomes more interesting with weaker models or harder tasks. *** ## 8. Supervisor (Autonomous Delegation) The LLM autonomously selects which sub-agent to invoke each iteration. ```java theme={null} AtomicInteger reviewCalls = new AtomicInteger(); AtomicInteger editCalls = new AtomicInteger(); Step review = Step.named("review", (ctx, in) -> { reviewCalls.incrementAndGet(); return chat.prompt() .user("Review this text and suggest one improvement: " + in) .call().content(); }); Step edit = Step.named("edit", (ctx, in) -> { editCalls.incrementAndGet(); return chat.prompt() .user("Edit this text to be more concise: " + in) .call().content(); }); Object result = Workflow.supervisor("text-improver", chat) .agents(review, edit) .until(ctx -> ctx.get(AgentContext.ITERATION_COUNT).orElse(0) >= 3) .run("The very big and extremely large dragon was flying very high " + "up in the sky above the tall mountains."); assertThat(reviewCalls.get() + editCalls.get()).isGreaterThanOrEqualTo(3); ``` The supervisor generates a routing prompt from agent names and descriptions. Each iteration, the LLM picks the most appropriate agent for the current state of the text. Terminates after 3 iterations. *** ## 9. Sub-workflow Composition A `Workflow` implements `Step` — nest one workflow inside another. Context writes from the inner workflow propagate back to the outer automatically. ```java theme={null} static final ContextKey QUALITY_KEY = ContextKey.of("quality", String.class); static final ContextKey SENTIMENT_KEY = ContextKey.of("sentiment", String.class); // Inner step that writes to context via updateContext() class AnalyzeQualityStep implements Step { @Override public String name() { return "analyze-quality"; } @Override public String execute(AgentContext ctx, String input) { return chat.prompt() .user("Rate this text quality as HIGH, MEDIUM, or LOW: " + input) .call().content().strip(); } @Override public AgentContext updateContext(AgentContext ctx, String output) { return ctx.mutate().with(QUALITY_KEY, output).build(); } } // Sub-workflow: analyze text, then summarize Workflow analyzeAndSummarize = Workflow.define("analyze") .step(new AnalyzeQualityStep()) .then(Step.named("summarize", (ctx, in) -> chat.prompt() .user("Summarize in one sentence: " + in) .call().content())) .build(); // Outer workflow uses sub-workflow as a step, then reads its context writes AtomicReference capturedQuality = new AtomicReference<>(); String result = (String) Workflow.define("outer") .step(analyzeAndSummarize) // sub-workflow — context propagates back .then(Step.named("read-ctx", (ctx, in) -> { capturedQuality.set(ctx.get(QUALITY_KEY).orElse("missing")); return in; })) .run("The quick brown fox jumps over the lazy dog."); assertThat(capturedQuality.get()).isIn("HIGH", "MEDIUM", "LOW"); // written inside sub-workflow ✓ assertThat(result).isNotBlank(); ``` Sub-workflows can be used anywhere a step is accepted: `.then()`, `.branch()`, `.otherwise()`, `.onPass()`, `.onFail()`, and `.parallel()`. Nesting is unlimited. *** ## Testing Strategy These examples demonstrate the assertion pattern for LLM-backed tests: * **Shape, not exact equality** — `isNotBlank()`, `hasSize(2)`, correct type * **Content signals** — expected keywords present (e.g., "doctor" for medical routing) * **Routing correctness** — branch/gate took the right path * **Convergence** — loops terminate within bounds * **Low temperature** (0.3) — reduces variance for test stability ## Run the Examples ```bash theme={null} git clone https://github.com/markpollack/workflow-dsl-examples.git cd workflow-dsl-examples export OPENAI_API_KEY=sk-... ./mvnw exec:java -pl module-01-sequential ``` Each module runs against real GPT-4.1 calls. See the [tutorial](/docs/agent-workflow/tutorial) for a guided walkthrough. # Getting Started with Agent Workflow Source: https://lab.pollack.ai/docs/agent-workflow/getting-started Compose steps into workflows with typed context, portable runtimes, and quality gates ## What is Agent Workflow? A workflow is a sequence of **steps**. Each step does one thing — calls an LLM, runs a function, invokes an external agent. Steps pass data through a shared **context** (typed key-value pairs). The workflow compiles to a **graph IR** — a pure data structure that decouples definition from execution. Three runtimes are available: `LocalStepRunner` (in-process, zero overhead), `CheckpointingStepRunner` (JDBC crash recovery), and `TemporalStepRunner` (distributed durable execution) — same workflow code, swap a single `@Bean`. ```java theme={null} Workflow.define("pr-review") .step(fetchDiff) .then(analyzeDiff) .gate(new JudgeGate(jury, 0.8)) .onPass(postComment) .onFail(revise) .end() .run(event); ``` ## Steps Steps are the building blocks. Each takes input, does work, produces output. ### Deterministic steps Pure Java — no LLM, no cost: ```java theme={null} Step fetchDiff = Step.named("fetch-diff", (ctx, in) -> { // Call GitHub API, return the diff as a string return gitHub.getPullRequest(in).getDiff(); }); ``` ### LLM steps Several flavors depending on what you're calling: | Step type | What it wraps | Typical duration | | ----------------- | ------------------------------------------------------------------------------------------------------ | ---------------- | | `ChatClientStep` | Single Spring AI `ChatClient` call | Seconds | | `ClaudeStep` | Full Claude CLI agent session (quick scripts, no trace capture) | Minutes | | `AgentClientStep` | External agent runtime with [trace capture](/docs/agent-workflow/trace-capture) (Claude, Gemini, etc.) | Minutes | | `A2AStep` | Remote agent via Agent-to-Agent protocol | Minutes | A `ClaudeStep` isn't a single API call — it runs a complete agentic loop internally. The workflow sees it as one step: ``` Workflow └── Step: FetchPR [deterministic — GitHub API call] └── Step: AnalyzeDiff [ClaudeStep — full agent loop internally] ├── LLM turn 1 → tool call → result ├── LLM turn 2 → tool call → result └── LLM turn N → finish └── Step: AssembleReport [deterministic — string formatting] ``` ### Creating steps with ChatClientStep For a single LLM call, `ChatClientStep` wraps a Spring AI `ChatClient`: ```java theme={null} ChatClient chat = ChatClient.builder(chatModel).build(); Step write = Step.named("write", (ctx, in) -> chat.prompt() .user("Write a 3-sentence story about: " + in) .call().content()); ``` ## Context Steps communicate through `AgentContext` — a typed key-value store that flows through the workflow: ```java theme={null} // Define typed keys static final ContextKey DIFF = ContextKey.of("diff", String.class); static final ContextKey RISK_SCORE = ContextKey.of("risk-score", Double.class); // Step reads from context and writes back Step assessRisk = Step.named("assess-risk", (ctx, in) -> { String diff = ctx.require(DIFF); // read upstream output double score = evaluateRisk(diff); // score becomes this step's output, available to downstream steps return score; }); ``` The context is immutable — each step gets a snapshot, mutations produce a new instance. Parallel branches receive independent snapshots and merge at join points. The framework auto-populates `Steps.outputOf("step-name")` after each step, so any downstream step can read any prior step's output by name. **Sub-workflow context propagation**: when a `Workflow` is used as a step inside another `Workflow`, its internal context mutations propagate back to the parent automatically. All `updateContext()` writes from nested steps are visible to downstream steps in the parent — no workarounds needed. ## Your First Workflow ```java theme={null} Step write = Step.named("write", (ctx, in) -> chat.prompt() .user("Write a 3-sentence story about: " + in) .call().content()); Step editForAudience = Step.named("edit-audience", (ctx, in) -> chat.prompt() .user("Rewrite for young adults. Return only the story: " + in) .call().content()); Step editForStyle = Step.named("edit-style", (ctx, in) -> chat.prompt() .user("Rewrite in a humorous style. Return only the story: " + in) .call().content()); String result = (String) Workflow.define("novel-creator") .step(write) .then(editForAudience) .then(editForStyle) .run("dragons and wizards"); ``` Output flows forward: `write` → `editForAudience` → `editForStyle`. Each step's output is the next step's input. ## The Graph IR The DSL doesn't execute directly — it builds a `WorkflowGraph`. This separation enables: * **Portable runtimes** — the IR decouples workflow definition from execution. Three runners ship today: `LocalStepRunner`, `CheckpointingStepRunner` (JDBC), and `TemporalStepRunner` (distributed) * **Tracing** — every step transition is recorded (`TraceRecorder`) * **Inspection** — the graph is pure data (nodes + edges), not opaque lambdas Control flow compiles to real graph structure: a branch is 4 nodes + 4 edges; a loop has a back-edge; parallel has fork/join nodes. All visible in traces. ## Prerequisites * Java 21+ * Spring AI 2.0 ```xml theme={null} io.github.markpollack workflow-flows 0.10.0 ``` ## What's Next Constructor injection, input chaining, context keys — 4 patterns for getting data into steps 10+ composable patterns — branch, loop, parallel, decision, gate, supervisor Crash recovery, checkpointing, and distributed execution 9 runnable integration tests validated against GPT-4.1 # Step Parameterization Source: https://lab.pollack.ai/docs/agent-workflow/parameterization How to get data into and out of steps — constructor injection, input chaining, context keys, metadata publishing, and mixed patterns The first question everyone asks: **"How do I get data into my steps?"** There are three levels, presented as a progression. Most users start at Level 1 and add complexity only when needed. | Level | Pattern | When you need it | | ------ | --------------------------------------------------------- | ------------------------------------------------------------------------------------ | | **1** | [Input chaining](#pattern-1-input-chaining) | Each step transforms the previous result (most common) | | **1b** | [Constructor injection](#pattern-2-constructor-injection) | Static config: model, threshold, API client | | **2** | [Context keys](#pattern-3-context-keys) | Step C needs step A's output, but step B sits between | | **3** | [Context writes](#pattern-5-context-writes-updatecontext) | Step publishes metadata (confidence, language, sources) alongside its primary output | ## Pattern 1: Constructor Injection Configuration known at build time — model, prompt template, threshold, API client. Same step class, different parameters, different behavior. ```java theme={null} static class TranslateStep implements Step { private final ChatClient chatClient; private final String targetLanguage; TranslateStep(ChatClient chatClient, String targetLanguage) { this.chatClient = chatClient; this.targetLanguage = targetLanguage; } @Override public Object execute(AgentContext ctx, Object input) { return chatClient.prompt() .user("Translate this to " + targetLanguage + ": " + input) .call().content(); } } // Same class, different config → different behavior Step toFrench = new TranslateStep(chat, "French"); Step toSpanish = new TranslateStep(chat, "Spanish"); Workflow.define("translate") .step(write) .then(toFrench) // or toSpanish — swap at build time .run("the sunrise over the mountains"); ``` **Use when**: Configuration is static — model selection, prompt templates, thresholds, API clients. Works exactly like a Spring `@Bean` with constructor injection. ## Pattern 2: Input Chaining Each step receives the previous step's output as its `input` parameter. No context keys, no configuration — just linear data flow. ```java theme={null} Workflow.define("chain") .step(Step.named("generate", (ctx, in) -> chat.prompt().user("Write a story about: " + in).call().content())) .then(Step.named("extract-character", (ctx, in) -> chat.prompt().user("Extract the main character's name: " + in).call().content())) .then(Step.named("describe-character", (ctx, in) -> chat.prompt().user("Describe a character named: " + in).call().content())) .run("a brave knight"); ``` **Use when**: The pipeline is linear — step B only needs step A's output. ## Pattern 3: Context Keys What if step C needs step A's output, but step B sits between them? The executor auto-propagates every step's output into `AgentContext` under `Steps.outputOf(stepName)`. Any downstream step can read any prior step's result by name. ```java theme={null} Workflow.define("context-keys") .step(Step.named("generate-story", (ctx, in) -> chat.prompt().user("Write a story about: " + in).call().content())) .then(Step.named("score-story", (ctx, in) -> chat.prompt().user("Rate this story 1-10: " + in).call().content())) .then(Step.named("summarize", (ctx, in) -> { // `in` is the score (from score-story via input chaining) // Read the story from generate-story via context Object story = ctx.get(Steps.outputOf("generate-story")).orElse("unknown"); return "Story: " + story + "\nScore: " + in; })) .run("a time-traveling cat"); ``` ### Typed context for structured data When steps produce structured data (lists, records, domain objects), downstream steps cast from the auto-propagated output: ```java theme={null} Step generateTopics = Step.named("generate-topics", (ctx, in) -> { String response = chat.prompt() .user("List 3 blog topics about " + in).call().content(); return List.of(response.split("\n")); }); Step selectTopic = Step.named("select-topic", (ctx, in) -> { List topics = (List) in; // typed via input chaining return chat.prompt() .user("Pick the most interesting: " + String.join(", ", topics)) .call().content(); }); Step writeDraft = Step.named("write-draft", (ctx, in) -> { String selected = (String) in; // Read the full topic list from step 1 via context Object allTopics = ctx.get(Steps.outputOf("generate-topics")).orElse("unknown"); return chat.prompt() .user("Write about: " + selected + "\nSelected from: " + allTopics) .call().content(); }); Workflow.define("blog-pipeline") .step(generateTopics) .then(selectTopic) .then(writeDraft) .run("artificial intelligence"); ``` **Use when**: A downstream step needs a non-adjacent prior step's output, or steps exchange structured data. ## Pattern 4: Mixed — All Three Together Real-world steps combine constructor config + input chaining + context state: ```java theme={null} static class ReviewStep implements Step { private final ChatClient chatClient; private final String reviewCriteria; // constructor: what to review for private final double passThreshold; // constructor: quality bar ReviewStep(ChatClient chatClient, String reviewCriteria, double passThreshold) { this.chatClient = chatClient; this.reviewCriteria = reviewCriteria; this.passThreshold = passThreshold; } @Override public Object execute(AgentContext ctx, Object input) { // input = content to review (from previous step) // reviewCriteria + passThreshold = constructor config // ctx = iteration count, prior outputs, workflow state int iteration = ctx.get(AgentContext.ITERATION_COUNT).orElse(0); double score = parseScore(chatClient.prompt() .user("Rate this for " + reviewCriteria + " (0-1): " + input) .call().content()); return "Review [" + reviewCriteria + ", iteration=" + iteration + "]: score=" + score + " (" + (score >= passThreshold ? "PASS" : "FAIL") + ")"; } } // Same class, different criteria Step clarityReview = new ReviewStep(chat, "clarity", 0.7); Step creativityReview = new ReviewStep(chat, "creativity", 0.6); ``` ## Pattern 5: Context Writes (`updateContext`) A step's primary job is to return a value — the category string for branching, the translated text for the next step. But sometimes a step also knows *metadata* that other steps need: the confidence score, the detected language, the list of sources used. Without `updateContext()`, the only way to pass all of this is to return a record — which forces every downstream step to know about that record type, killing reusability. With `updateContext()`, the step returns its primary output AND publishes metadata as typed context keys. ### Example: Classification with confidence and reasoning ```java theme={null} static class ClassifierStep implements Step { // Published constants — the step's "output contract" static final ContextKey CONFIDENCE = ContextKey.of("classifier.confidence", Double.class); static final ContextKey REASONING = ContextKey.of("classifier.reasoning", String.class); private final ChatClient chat; private double confidence; private String reasoning; ClassifierStep(ChatClient chat) { this.chat = chat; } @Override public String name() { return "classifier"; } @Override public Object execute(AgentContext ctx, Object input) { String response = chat.prompt() .user("Classify as 'medical' or 'legal'. " + "Reply: CATEGORY: \nCONFIDENCE: <0-1>\nREASONING: \n\n" + "Request: " + input) .call().content(); // Parse structured response String category = "unknown"; for (String line : response.split("\n")) { if (line.startsWith("CATEGORY:")) category = line.substring(9).strip().toLowerCase(); else if (line.startsWith("CONFIDENCE:")) { try { confidence = Double.parseDouble(line.substring(11).strip()); } catch (NumberFormatException e) { /* keep default */ } } else if (line.startsWith("REASONING:")) reasoning = line.substring(10).strip(); } return category; // primary output — for branch routing } @Override public AgentContext updateContext(AgentContext ctx, Object output) { return ctx.mutate() .with(CONFIDENCE, confidence) .with(REASONING, reasoning) .build(); } } ``` The classifier returns `"medical"` as its primary output (used by the branch). The confidence and reasoning are published as side-channel metadata. Any downstream step reads them without knowing anything about the classifier's internals: ```java theme={null} Workflow.define("classify-pipeline") .step(new ClassifierStep(chat)) .branch(output -> "medical".equals(output)) .then(Step.named("medical", (ctx, in) -> "Medical advice provided")) .otherwise(Step.named("legal", (ctx, in) -> "Legal advice provided")) .then(Step.named("audit", (ctx, in) -> { // Generic audit step — reads metadata by key, doesn't know ClassifierStep double conf = ctx.get(ClassifierStep.CONFIDENCE).orElse(-1.0); String reason = ctx.get(ClassifierStep.REASONING).orElse("none"); return in + " [confidence=" + conf + ", reasoning=" + reason + "]"; })) .run("I broke my leg, what should I do?"); ``` ### Example: Language detection as side-channel ```java theme={null} static class DetectAndTranslateStep implements Step { static final ContextKey DETECTED_LANGUAGE = ContextKey.of("translate.detectedLanguage", String.class); private final ChatClient chat; private String detectedLang; DetectAndTranslateStep(ChatClient chat) { this.chat = chat; } @Override public String name() { return "detect-and-translate"; } @Override public Object execute(AgentContext ctx, Object input) { // ... call LLM, parse "LANGUAGE:" and "TRANSLATION:" from response ... return translation; // primary output — the English text } @Override public AgentContext updateContext(AgentContext ctx, Object output) { return ctx.mutate().with(DETECTED_LANGUAGE, detectedLang).build(); } } // Downstream step reads the language without coupling to the translator Step audit = Step.named("audit", (ctx, in) -> { String lang = ctx.get(DetectAndTranslateStep.DETECTED_LANGUAGE).orElse("unknown"); return "Translated from " + lang + ": " + in; }); ``` **Use when**: A step produces a primary result AND secondary data (confidence, language, token counts, source lists). The step class owns `ContextKey` constants as its published output contract. Downstream steps read by key — no coupling to the producing step's record type. Most users never need `updateContext()`. Start with input chaining (Pattern 1). Add `Steps.outputOf()` when you need non-adjacent data (Pattern 3). Reach for `updateContext()` only when a step genuinely produces metadata that should travel separately from its primary output. ## Quick Reference | Level | Pattern | Data source | Known when | Example | | ------ | ------------------ | ------------------------------- | ---------- | ----------------------------- | | **1** | **Input chaining** | Previous step's output | Runtime | Linear pipelines | | **1b** | **Constructor** | Step constructor args | Build time | Model, threshold, API client | | **2** | **Context keys** | Any prior step's output by name | Runtime | Non-adjacent steps | | **3** | **Context writes** | Step publishes typed metadata | Runtime | Confidence, language, sources | | **—** | **Mixed** | All of the above | Both | Real-world steps | ## Related Sequential, parallel, gate, loop, branch, and more Runnable integration tests including context writes # Trace Capture Source: https://lab.pollack.ai/docs/agent-workflow/trace-capture Capture per-step JSONL trace files and wire them through the workflow journal Workflow tracing records *which steps ran* via `StepTransition`. Trace capture goes deeper — it records *what happened inside each step*: every tool call, thinking block, token count, and cost. The trace is written to a JSONL file during execution, and the file path flows through the workflow journal so analysis tools can find it. ## When you need trace capture * **Markov analysis** — fingerprint an agent's behavioral patterns across runs * **Cost attribution** — break down per-step token usage and cost * **Debugging** — replay exactly what the agent saw, thought, and did * **Regression detection** — compare traces across code changes ## Setup Trace capture requires two things: an agent model that writes trace files, and a workflow client that propagates the path. ### 1. Configure traceDir on ClaudeAgentModel The `ClaudeAgentModel` from [agent-client](https://central.sonatype.com/artifact/io.github.markpollack/agent-claude) writes a JSONL trace file per `call()` when `traceDir` is set: ```java theme={null} ClaudeAgentModel model = ClaudeAgentModel.builder() .traceDir(Path.of("traces")) .build(); ``` Each call produces a file like `traces/agent-run-20260528-143000-a1b2c3d4.jsonl` containing every message in the session. ### 2. Bridge to a trace-aware AgentClient The workflow-flows `AgentClient` is a `@FunctionalInterface` that returns text. To carry trace metadata, override `executeForResult()`: ```java theme={null} var coreClient = AgentClient.create(model); io.github.markpollack.workflow.flows.steps.AgentClient workflowClient = new io.github.markpollack.workflow.flows.steps.AgentClient() { @Override public String execute(String prompt, AgentContext ctx) { return executeForResult(prompt, ctx).text(); } @Override public ExecutionResult executeForResult(String prompt, AgentContext ctx) { AgentClientResponse response = coreClient.run(prompt); String tracePath = (String) response.getMetadata().get("tracePath"); return new ExecutionResult(response.getResult(), tracePath); } }; ``` Plain lambdas still work — `executeForResult()` defaults to calling `execute()` with a null trace path. ### 3. Use AgentClientStep in a workflow ```java theme={null} AgentClientStep fixStep = AgentClientStep.of(workflowClient, "Fix: {input}"); AgentClientStep verifyStep = AgentClientStep.of(workflowClient, "Verify the fix: {input}"); TraceRecorder recorder = TraceRecorder.inMemory(); WorkflowExecutor executor = new WorkflowExecutor(recorder); String result = Workflow.define("remediate") .withExecutor(executor) .step(fixStep) .then(verifyStep) .run("failing test in AuthService"); ``` Each `AgentClientStep` gets its own trace file. The path flows through to `StepTransition`: ```java theme={null} List trace = recorder.getTrace(runId); for (StepTransition t : trace) { if (t.tracePath() != null) { System.out.println(t.toStep() + " → " + t.tracePath()); } } // AgentClientStep → /abs/path/traces/agent-run-20260528-143000-a1b2c3d4.jsonl // AgentClientStep → /abs/path/traces/agent-run-20260528-143500-e5f6g7h8.jsonl ``` ## How it works The trace path flows through four layers: ``` AgentClient.executeForResult() → ExecutionResult(text, tracePath) AgentClientStep.updateContext() → sets AgentContext.TRACE_PATH WorkflowExecutor.recordTransition() → reads TRACE_PATH, clears it, records StepTransition TraceRecorder → stores/persists the transition ``` The executor clears `TRACE_PATH` from context after each step so deterministic steps don't inherit a stale path. ## Journal integration When using `workflow-journal`, trace paths appear in `WorkflowStepEvent` and are included in the journal's JSON output: ```java theme={null} Journal.configure(new JsonFileStorage(journalDir)); WorkflowJournal.registerEventType(); try (Run run = Journal.run("remediate-experiment").start()) { WorkflowExecutor executor = new WorkflowExecutor( new LocalStepRunner(), WorkflowJournal.forRun(run)); // ... run workflow } ``` The journal event includes `tracePath` when present: ```json theme={null} { "type": "workflow_step", "stepName": "AgentClientStep", "nodeType": "AGENT", "stepDurationMs": 12000, "tokensUsed": 3200, "costUsd": 0.048, "tracePath": "/abs/path/traces/agent-run-20260528-143000-a1b2c3d4.jsonl" } ``` ## JDBC persistence `JdbcTraceRecorder` stores trace paths in the `trace_path` column of `step_transitions`: ```java theme={null} JdbcTraceRecorder recorder = new JdbcTraceRecorder(dataSource); List trace = recorder.getTrace("run-1"); // Find all trace files for a run List traceFiles = trace.stream() .map(StepTransition::tracePath) .filter(Objects::nonNull) .toList(); ``` ## ClaudeStep vs AgentClientStep | | ClaudeStep | AgentClientStep | | --------------- | -------------------------------- | --------------------------------- | | Execution | CLI subprocess (`claude -p`) | In-process via `ClaudeAgentModel` | | Trace capture | Not available (text-only output) | Full JSONL trace files | | Token/cost data | Discarded at process boundary | Available in `providerFields` | | Use case | Quick scripts, prototyping | Experiments, production workflows | For any workflow where you need to analyze what the agent did — use `AgentClientStep`. ## Maven coordinates ```xml theme={null} io.github.markpollack workflow-flows 0.10.0 io.github.markpollack agent-client-core 0.29.0 io.github.markpollack agent-claude 0.29.0 ``` Or use the [AgentWorks BOM](/projects/agentworks-bom) (1.1.0+) for managed versions. ## Related JdbcTraceRecorder, CheckpointingStepRunner StepTransition, TraceRecorder, WorkflowExecutor # Tutorial: Build a Workflow Source: https://lab.pollack.ai/docs/agent-workflow/tutorial Build a multi-step AI workflow from a single step to a supervised agent pipeline ## What You'll Build A series of workflows that progressively introduce every DSL primitive: sequential pipelines, conditional branching, error recovery, loops, parallel execution, LLM-driven routing, quality gates, and supervised agent delegation. Each step is a real integration test validated against GPT-4.1. ## Prerequisites * Java 21+ * An OpenAI API key (`OPENAI_API_KEY` environment variable) * Agent Workflow 0.10.0: ```xml theme={null} io.github.markpollack workflow-flows 0.10.0 ``` ## Setup All examples share a `ChatClient` configured for GPT-4.1 with low temperature for test stability: ```java theme={null} String apiKey = System.getenv("OPENAI_API_KEY"); OpenAiApi api = OpenAiApi.builder().apiKey(apiKey).build(); OpenAiChatModel model = OpenAiChatModel.builder() .openAiApi(api) .defaultOptions(OpenAiChatOptions.builder() .model("gpt-4.1") .maxTokens(1024) .temperature(0.3) .build()) .build(); ChatClient chat = ChatClient.builder(model).build(); ``` *** ## Step 1: Define a Step and Chain a Pipeline A `Step` is the building block — a named function that takes context and input, produces output. Chain steps with `.then()` and each step's output flows into the next. ```java theme={null} Step write = Step.named("write", (ctx, in) -> chat.prompt() .user("You are a creative writer. Write a 3-sentence story about: " + in) .call().content()); Step editForAudience = Step.named("edit-audience", (ctx, in) -> chat.prompt() .user("Rewrite this story for young adults. Return only the story: " + in) .call().content()); Step editForStyle = Step.named("edit-style", (ctx, in) -> chat.prompt() .user("Rewrite this story in a humorous style. Return only the story: " + in) .call().content()); String result = (String) Workflow.define("novel-creator") .step(write) .then(editForAudience) .then(editForStyle) .run("dragons and wizards"); ``` Three LLM calls in sequence: write a story, rewrite for audience, rewrite for style. The `Workflow.define()` + `.run()` pattern compiles the graph and executes it in one call. [View source: SequentialDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-01-sequential/src/main/java/io/github/markpollack/workflow/tutorial/module01/SequentialDemo.java) *** ## Step 2: Branch on Classification Route to different steps based on a predicate applied to the previous step's output. ```java theme={null} Step classify = Step.named("classify", (ctx, in) -> chat.prompt() .user("Classify this as either 'medical' or 'legal'. " + "Reply with exactly one word: " + in) .call().content().strip().toLowerCase()); Step medicalExpert = Step.named("medical", (ctx, in) -> chat.prompt() .user("You are a medical expert. Briefly advise on: " + in) .call().content()); Step legalExpert = Step.named("legal", (ctx, in) -> chat.prompt() .user("You are a legal expert. Briefly advise on: " + in) .call().content()); String result = (String) Workflow.define("category-router") .step(classify) .branch(output -> "medical".equals(output)) .then(medicalExpert) .otherwise(legalExpert) .run("I broke my leg, what should I do?"); ``` The `.strip().toLowerCase()` on the classify output matters — LLMs sometimes return trailing whitespace or mixed case. The `branch()` predicate is a plain Java `Predicate`, so you can test any condition. [View source: BranchDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-02-branch/src/main/java/io/github/markpollack/workflow/tutorial/module02/BranchDemo.java) *** ## Step 3: Handle Errors Route exceptions to a recovery step instead of crashing the workflow. The recovery step's output flows into the next step as if the risky step had succeeded. ```java theme={null} Step riskyStep = Step.named("risky", (ctx, in) -> { if (((String) in).contains("bad")) { throw new IllegalArgumentException("Bad input detected"); } return chat.prompt() .user("Process this: " + in) .call().content(); }); Step recovery = Step.named("recovery", (ctx, in) -> chat.prompt() .user("The previous step failed. " + "Generate a safe default response for: " + in) .call().content()); Step finalStep = Step.named("finalize", (ctx, in) -> "Final: " + in); String result = (String) Workflow.define("error-recovery") .step(riskyStep) .onError(IllegalArgumentException.class, recovery) .then(finalStep) .run("bad input"); // result starts with "Final:" — the workflow continued through recovery ``` The `.onError()` clause is type-specific — you can attach different recovery steps for different exception types. The workflow graph wires the recovery path at compile time, not at catch time. [View source: ErrorRecoveryDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-03-error-recovery/src/main/java/io/github/markpollack/workflow/tutorial/module03/ErrorRecoveryDemo.java) *** ## Step 4: Loop Until Quality Converges Iterate a block of steps until a predicate on the output is satisfied. This is the most complex primitive — LLM score parsing needs care. ```java theme={null} AtomicInteger iterations = new AtomicInteger(0); Step editor = Step.named("editor", (ctx, in) -> chat.prompt() .user("Write a very short (2-sentence) extremely funny joke about dragons. " + "Be hilarious.") .call().content()); Step scorer = Step.named("scorer", (ctx, in) -> { iterations.incrementAndGet(); String response = chat.prompt() .user("Rate this text for humor on a scale of 0.0 to 1.0. " + "Reply with ONLY a decimal number, nothing else: " + in) .call().content().strip(); try { return Double.parseDouble(response); } catch (NumberFormatException e) { var matcher = java.util.regex.Pattern.compile("\\d+\\.\\d+").matcher(response); return matcher.find() ? Double.parseDouble(matcher.group()) : 0.0; } }); Object result = Workflow.define("humor-loop") .repeatUntilOutput(score -> score instanceof Double d && d >= 0.6) .step(editor) .step(scorer) .end() .run("A dragon walked into a bar."); ``` The loop alternates between `editor` (generate) and `scorer` (evaluate) until the score reaches the threshold. GPT-4.1 returns clean decimal numbers with the "Reply with ONLY a decimal number" prompt — the regex fallback is there for other models. [View source: LoopDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-04-loop/src/main/java/io/github/markpollack/workflow/tutorial/module04/LoopDemo.java) *** ## Step 5: Fan Out in Parallel Run steps concurrently and collect results into a list, ordered to match step order. ```java theme={null} Step findMeals = Step.named("find-meals", (ctx, in) -> chat.prompt() .user("Suggest 3 meals for a " + in + " evening. " + "Just list the meal names, one per line.") .call().content()); Step findMovies = Step.named("find-movies", (ctx, in) -> chat.prompt() .user("Suggest 3 movies for a " + in + " evening. " + "Just list the movie titles, one per line.") .call().content()); @SuppressWarnings("unchecked") List results = (List) Workflow.define("evening-planner") .parallel(findMeals, findMovies) .run("romantic"); // results.get(0) = meal suggestions // results.get(1) = movie suggestions ``` Both LLM calls execute concurrently. You can pass any number of steps to `.parallel()` — they all receive the same input and their outputs are collected in order. [View source: ParallelDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-05-parallel/src/main/java/io/github/markpollack/workflow/tutorial/module05/ParallelDemo.java) *** ## Step 6: LLM-Driven Routing Let the LLM choose which step to execute. Unlike `branch()` (predicate-based), `decision()` gives the LLM a menu of labeled options and it picks one. ```java theme={null} Step summarize = Step.named("summarize", (ctx, in) -> chat.prompt() .user("Summarize this in one sentence: " + in) .call().content()); Step translate = Step.named("translate", (ctx, in) -> chat.prompt() .user("Translate this to French: " + in) .call().content()); String result = (String) Workflow.define("decision-router") .decision(chat) .option("summarize", summarize) .option("translate", translate) .end() .run("The quick brown fox jumps over the lazy dog. " + "This is a classic English pangram used for testing."); ``` The DSL generates a routing prompt from the option names. The LLM returns a clean single-word label and the corresponding step executes. Use `decision()` when the routing logic is semantic (the LLM needs to understand the input to choose) rather than structural (a simple predicate suffices). [View source: DecisionDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-06-decision/src/main/java/io/github/markpollack/workflow/tutorial/module06/DecisionDemo.java) *** ## Step 7: Quality Gate Evaluate output quality and route to pass or fail paths. The gate is a function that returns `GateDecision.PASS` or `GateDecision.FAIL`. ```java theme={null} Gate qualityGate = (ctx, output) -> { String response = chat.prompt() .user("Rate this text for quality on a scale of 0.0 to 1.0. " + "Reply with ONLY a decimal number: " + output) .call().content().strip(); double score; try { score = Double.parseDouble(response); } catch (NumberFormatException e) { var matcher = java.util.regex.Pattern.compile("\\d+\\.\\d+").matcher(response); score = matcher.find() ? Double.parseDouble(matcher.group()) : 0.0; } return score >= 0.7 ? GateDecision.PASS : GateDecision.FAIL; }; Step generate = Step.named("generate", (ctx, in) -> chat.prompt() .user("Write a well-crafted 2-sentence story about: " + in) .call().content()); Step approve = Step.named("approve", (ctx, in) -> "APPROVED: " + in); Step reject = Step.named("reject", (ctx, in) -> "REJECTED: " + in); String result = (String) Workflow.define("gated-pipeline") .step(generate) .gate(qualityGate) .onPass(approve) .onFail(reject) .end() .run("a heroic knight"); ``` Gates are the integration point for [Agent Judge](/projects/agent-judge) — replace the lambda gate with a `JudgeGate` backed by a jury for multi-judge evaluation with voting strategies. [View source: GateDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-07-gate/src/main/java/io/github/markpollack/workflow/tutorial/module07/GateDemo.java) *** ## Step 8: Supervisor The LLM autonomously selects which sub-agent to invoke each iteration, terminating when a condition is met. ```java theme={null} Step review = Step.named("review", (ctx, in) -> chat.prompt() .user("Review this text and suggest one improvement: " + in) .call().content()); Step edit = Step.named("edit", (ctx, in) -> chat.prompt() .user("Edit this text to be more concise: " + in) .call().content()); Object result = Workflow.supervisor("text-improver", chat) .agents(review, edit) .until(ctx -> ctx.get(AgentContext.ITERATION_COUNT).orElse(0) >= 3) .run("The very big and extremely large dragon was flying very high " + "up in the sky above the tall mountains."); ``` The supervisor generates a routing prompt from agent names. Each iteration, the LLM picks the most appropriate agent for the current state. The `.until()` predicate reads from context — `ITERATION_COUNT` is automatically maintained, but you can use any context key. [View source: SupervisorDemo.java](https://github.com/markpollack/workflow-dsl-examples/blob/main/module-08-supervisor/src/main/java/io/github/markpollack/workflow/tutorial/module08/SupervisorDemo.java) *** ## Runnable Code Every step in this tutorial has a corresponding runnable module in the [workflow-dsl-examples](https://github.com/markpollack/workflow-dsl-examples) repository. Clone it and run any module with `./mvnw exec:java -pl module-NN-name`. ```bash theme={null} git clone https://github.com/markpollack/workflow-dsl-examples.git cd workflow-dsl-examples export OPENAI_API_KEY=sk-... ./mvnw exec:java -pl module-01-sequential ``` ## What's Next Declarative workflows with @Agent, @ExceptionHandler, and AgentRegistry 4 patterns for getting data into steps Crash recovery with CheckpointingStepRunner and Temporal Complete reference with all 9 patterns plus sub-workflow composition # What's New Source: https://lab.pollack.ai/docs/agent-workflow/whats-new Release highlights for Agent Workflow ## 0.10.0 (2026-06-15) * Spring AI 2.0.0 GA (clears CVE-2026-41712; spring-ai-client-chat 2.0.0) ## 0.9.0 (2026-06-06) * Jackson 2.21.2 alignment, journal 1.4.0 + judge 0.12.0 pins, release 0.9.0 * Migrate judge integration to io.github.markpollack namespace * Add journal entry for trace path wiring session ## 0.8.0 **Trace file capture through the workflow journal.** `AgentClientStep` now propagates trace file paths from the underlying agent model into the workflow's `StepTransition` and journal events. This closes the gap between "the agent ran" and "here's exactly what the agent did" — every tool call, token count, and thinking block is preserved in a JSONL trace file, and the path to that file is recorded in the workflow journal. ### AgentClientStep trace wiring When the backing `AgentClient` implementation returns a trace path via `executeForResult()`, `AgentClientStep` writes it to `AgentContext.TRACE_PATH` and the executor propagates it through to `StepTransition.tracePath()`: ```java theme={null} // Trace-aware client bridges agent-client to workflow-flows AgentClient traceClient = new AgentClient() { @Override public String execute(String prompt, AgentContext ctx) { return executeForResult(prompt, ctx).text(); } @Override public ExecutionResult executeForResult(String prompt, AgentContext ctx) { AgentClientResponse response = coreClient.run(prompt); String tracePath = (String) response.getMetadata().get("tracePath"); return new ExecutionResult(response.getResult(), tracePath); } }; AgentClientStep step = AgentClientStep.of(traceClient, "Fix the bug: {input}"); ``` In a multi-step workflow, each AI step gets its own trace file. Deterministic steps pass through with no trace. Analysis scripts iterate over journal events to find all trace files for a run: ``` Workflow: remediate step: trivy-scan → tracePath: null (deterministic) step: claude-fix → tracePath: /traces/fix-20260528-143000.jsonl step: mvn-test → tracePath: null (deterministic) step: claude-verify → tracePath: /traces/verify-20260528-143500.jsonl ``` ### What changed * `AgentClient.executeForResult()` — new default method returning `ExecutionResult(text, tracePath)`. Backward compatible: lambdas still work, trace path is null. * `AgentClientStep` — calls `executeForResult()`, writes `TRACE_PATH` to context via `updateContext()` * `StepTransition` and `WorkflowStepEvent` — carry optional `tracePath` field * `JdbcTraceRecorder` — new `trace_path` column in `step_transitions` table * `AgentContext.TRACE_PATH` — new well-known context key * `AgentContext.Builder.without()` — remove a key from context (used internally to clear per-step trace paths) * `ClaudeStep` javadoc — now recommends `AgentClientStep` for experiments needing trace capture See [Trace Capture](/docs/agent-workflow/trace-capture) for the full guide. *** ## 0.7.0 **Managed Agents as a step runtime.** The new `ManagedAgentStep` delegates workflow steps to [Anthropic's Managed Agents API](https://docs.anthropic.com/en/api/overview). The workflow graph stays in charge — but specific steps can run in Anthropic's cloud sandbox with full tool access (bash, file I/O, web search). This proves the core thesis: **workflow is portable, execution substrate is pluggable.** The same workflow definition can run steps locally, via Temporal, or in Anthropic's hosted infrastructure — swap a single step, zero changes to the graph. ### ManagedAgentStep Reference a pre-created agent and environment: ```java theme={null} ManagedAgentStep step = ManagedAgentStep.of(agentId, environmentId) .name("remediation-agent") .timeout(Duration.ofMinutes(10)); String result = step.execute(ctx, "Fix the failing test in AuthService.java"); ``` Or provision a new agent inline with the default toolset: ```java theme={null} ManagedAgentStep step = ManagedAgentStep.create( "claude-sonnet-4-6", "You are a code remediation agent. Fix failing tests.", environmentId); ``` Use it in a workflow like any other step: ```java theme={null} var workflow = WorkflowGraph.builder() .step("analyze", analyzeStep) .step("remediate", ManagedAgentStep.of(agentId, envId).name("fix")) .step("validate", validateStep) .build(); ``` **Design choices:** * Uses the official [`com.anthropic:anthropic-java`](https://github.com/anthropics/anthropic-sdk-java) SDK directly — no wrapper layer * Follows the `A2AStep` immutable pattern: factory methods, copier methods for `name()` / `timeout()`, `SessionRunner` functional interface for testability * One session per `execute()` call — agents are reused, sessions are ephemeral * Stream-first SSE pattern: opens the event stream before sending the user message, collects `agentMessage` text blocks, breaks on `sessionStatusIdle` or `sessionStatusTerminated` **Dependency** (optional — only needed if you use `ManagedAgentStep`): ```xml theme={null} com.anthropic anthropic-java 2.34.0 ``` ### Step runtime landscape With 0.7.0, Agent Workflow supports five step runtimes: | Step | Runtime | Use case | | ---------------------- | ------------------- | ------------------------------------------------------------------------ | | `Steps.of(fn)` | Local JVM | Deterministic logic, API calls | | `ChatClientStep` | Spring AI | Single LLM call, structured output | | `ClaudeStep` | Claude CLI | Quick scripts (no trace capture) | | `AgentClientStep` | agent-client | Full agent loop with [trace capture](/docs/agent-workflow/trace-capture) | | `A2AStep` | A2A protocol | Remote agent delegation | | **`ManagedAgentStep`** | **Anthropic cloud** | **Hosted agent with sandbox** | *** ## 0.6.0 Initial public release on Maven Central. See [Getting Started](/docs/agent-workflow/getting-started) and the [Tutorial](/docs/agent-workflow/tutorial). # Wiring Complex Pipelines Source: https://lab.pollack.ai/docs/agent-workflow/wiring How to structure the constructor and Spring configuration for workflows with many collaborating steps ## The Problem A real pipeline has many steps — fetch context, rebase, run tests, run AI assessments, judge output, generate a report. Passing every leaf step to a single top-level constructor produces an argument list that's hard to read, hard to test, and hard to explain: ```java theme={null} // 13 arguments — too much to take in at once public PrReviewDslWorkflow( FetchPrContextStep fetchPrContext, RebaseStep rebaseStep, ConflictDetectionStep conflictDetection, RunTestsStep runTests, FixAndRetestStep fixAndRetestStep, CleanupStep cleanupStep, BuildGate buildGate, VersionPatternStep versionPatternStep, Step assessCodeQuality, Step assessBackport, QualityJudgeStep qualityJudgeStep, AssembleReportStep assembleReportStep, GenerateReportStep generateReport) ``` The fix is not cosmetic. The constructor is wrong at the **level of abstraction** — it describes leaves when it should describe structure. ## The Pattern Pre-assemble sub-workflows as named Spring beans. The top-level workflow takes phases, not leaves: ```java theme={null} // 4 arguments — each one a named structural unit public PrReviewDslWorkflow( Workflow contextPhase, // fetch → rebase → conflict → tests → cleanup BuildGate buildGate, Workflow assessAndReport, // T0 pass: version check → AI → report Workflow earlyReport) // T0 fail: assemble → report ``` The constructor body becomes a pure description of how the phases connect — no wiring, no service lookups: ```java theme={null} this.pipeline = Workflow.define("pr-review") .step(contextPhase) .gate(buildGate) .onPass(assessAndReport) .onFail(earlyReport) .end() .build(); ``` ## The Factory The Spring `@Configuration` class is the wiring hub. It assembles each phase as a `@Bean`. Because `Workflow` implements `Step`, sub-workflows compose directly into parent workflows with no adapter needed. ```java theme={null} @Configuration public class DslWorkflowConfig { @Bean Workflow contextPhase( FetchPrContextStep fetch, RebaseStep rebase, ConflictDetectionStep conflict, RunTestsStep tests, FixAndRetestStep fix, CleanupStep cleanup) { return Workflow.define("context-phase") .step(fetch) .then(rebase) .then(conflict) .then(tests) .then(fix) .then(cleanup) .build(); } @Bean Workflow aiAssessment( Step assessCodeQuality, Step assessBackport) { return Workflow.define("ai-assessment") .step(new ExtractPrContextStep()) .parallel(assessCodeQuality, assessBackport) .build(); } @Bean Workflow assessAndReport( VersionPatternStep versionPattern, Workflow aiAssessment, QualityJudgeStep qualityJudge, AssembleReportStep assembleReport, GenerateReportStep generateReport) { return Workflow.define("assess-and-report") .step(versionPattern) .then(aiAssessment) .then(qualityJudge) .then(assembleReport) .then(generateReport) .build(); } @Bean Workflow earlyReport( AssembleReportStep assembleReport, GenerateReportStep generateReport) { return Workflow.define("early-report") .step(assembleReport) .then(generateReport) .build(); } @Bean PrReviewDslWorkflow prReviewDslWorkflow( Workflow contextPhase, BuildGate buildGate, Workflow assessAndReport, Workflow earlyReport) { return new PrReviewDslWorkflow(contextPhase, buildGate, assessAndReport, earlyReport); } } ``` The factory assembles the pieces; the workflow describes only their arrangement. Each bean is independently inspectable and testable. ## Why This Structure **Separation of concerns**: the workflow is a structural description — it answers "what runs when." The factory is the wiring layer — it answers "what object gets what dependency." Mixing them produces the 13-argument constructor. **Independent testability**: each sub-workflow is a `Workflow` bean that can be tested in isolation with a minimal set of mock steps. You don't need to construct all 13 collaborators to test the AI assessment phase: ```java theme={null} // Test the AI assessment phase in isolation Workflow assessAndReport = new PrReviewConfig() .assessAndReport(versionPattern, mockAssessCode, mockAssessBackport, qualityJudge, assembleReport, generateReport); Path result = assessAndReport.execute(ctx, prContext); assertThat(result).exists(); ``` **Readable at every level**: a reader of `PrReviewDslWorkflow` sees the four phases and the gate. A reader of `PrReviewConfig.assessAndReport()` sees the five steps. Neither method is overwhelmed by the other's details. ## What Other Frameworks Do This is the standard pattern across Java workflow and batch frameworks: * **LangChain4j** — `@SequenceAgent(subAgents = {A.class, B.class})` references agents (phases), not the services inside them. The orchestrator doesn't know how agent `A` is wired. * **Google ADK Java** — `SequentialAgent.builder().addSubAgent(parallelAgent).build()` where `parallelAgent` is already assembled. Leaf services stay inside sub-agents. * **Spring Batch** — `Job` references `Step` beans. A `Step` may contain an `ItemReader`, `ItemProcessor`, and `ItemWriter`, but the job definition never sees those — it sees only the step. The consistent rule: **the top-level orchestrator describes structure; the factory describes wiring.** ## Summary | Concern | Where it lives | | ----------------------------------- | ----------------------------------- | | Pipeline structure (what runs when) | `PrReviewDslWorkflow` constructor | | Sub-workflow assembly | `@Bean` methods in `@Configuration` | | Leaf step construction | `@Bean` methods in `@Configuration` | | Service/judge injection | `@Bean` method parameters | The workflow constructor should be readable to anyone who wants to understand the pipeline. The configuration class should be readable to anyone who wants to understand how dependencies flow in. Full vocabulary of composable primitives Sub-workflow composition, AgentContext, StepRunner # What's New Source: https://lab.pollack.ai/docs/agentworks-bom/whats-new Release notes for AgentWorks BOM — auto-generated from git history ## 1.16.0 (2026-08-22) * Repin agent-client family 0.25.0 → 0.28.0, adding the Grok and Antigravity providers to the managed set * Repin Agent Journal 1.6.0 → 1.7.0, Agent Hooks 0.6.4 → 0.7.0, Agent Judge 0.13.0 → 0.15.0, Agent Sandbox 0.9.3 → 0.10.0, Agent Bench 0.4.0 → 0.6.0, Agent Experiment 0.5.0 → 0.6.0, Agent Memory 0.3.0 → 0.4.0, and Claude Agent SDK 1.4.0 → 1.5.0 * Repin the acp-java family 0.14.0 → 0.15.0 * Drop `loopy` from the managed set; it ships an executable distribution and is depended on directly ## 1.15.0 (2026-07-01) * Repin agent-client family 0.24.0 → 0.25.0 — fixes Spring Boot auto-configuration for all providers (org migration had left the registration files at the old org.springaicommunity package → ClassNotFoundException at boot); all other members unchanged; gate 9/9 ## 1.14.0 (2026-07-01) * Repin agent-client family 0.23.0 → 0.24.0 — compatibility fix so agent-claude builds against journal 1.6.0 (journal 1.5.0 relocated TraceContentMode, breaking Claude trace-wiring on BOM 1.12.0/1.13.0); all other members unchanged; gate 9/9 ## 1.13.0 (2026-07-01) * Repin agent-journal to 1.6.0 (journal-core, claude-code-capture, gemini-cli-capture) — first-class capture primitives (slice 1), cost-metering fix, A5 schema-version header; gate 9/9 ## 1.12.0 (2026-06-17) * Repin agent-journal 1.5.0 + add gemini-cli-capture as a managed member; gate default -> 1.11.0 * CI: gate the BOM release on bom-verification (pre-publish 9/9 + post-publish FINAL\_PROOF) ## 1.11.0 (2026-06-15) * agentworks-bom 1.11.0: repin source-fixed members (CVE remediation) ## 1.10.0 (2026-06-15) * agentworks-bom 1.10.0: claude-code-sdk 1.4.0 + workflow 0.10.0; spring-ai-bom 2.0.0 backstop ## 1.9.0 (2026-06-14) * agentworks-bom 1.9.0: agent-client 0.22.0, Jackson 3 + reactor convergence; CI-releasable bom-verification ## 1.8.0 (2026-06-11) * BOM 1.8.0: acp 0.14.0 (clears the 0.13.0 deferral); member set otherwise unchanged from 1.7.0 ## 1.7.0 (2026-06-07) * BOM 1.7.0: agent-client 0.21.0 (effort options, provider parity, claude-code-sdk 1.3.0); member set otherwise unchanged from 1.6.0; acp remains 0.12.0 pending CI fix ## 1.6.0 (2026-06-06) * BOM 1.6.0 — aligned member releases, Jackson 2.21.2 convergence, bom-verification quality gate * Pin workflow modules to 0.9.0-SNAPSHOT (markpollack judge namespace) ## 1.5.0 (2026-06-04) * Bump journal-core and claude-code-capture to 1.3.0 * Add test infrastructure management and workflow-journal to BOM ## 1.4.0 (2026-05-29) * Add BSL 1.1 license * Remove CLAUDE.md from tracking and update .gitignore * Bump experiment-core and experiment-claude to 0.4.0 ## 1.3.0 (2026-05-29) * Bump claude-code-sdk to 1.2.0 (fix sync client hang after max-turns) ## 1.2.0 (2026-05-29) * Bump acp-\* from 0.11.0 to 0.12.0 ## 1.1.0 (2026-05-29) * Update agent-workflow to 0.8.0 and agent-client to 0.19.0 ## 1.0.13 (2026-05-28) * Bump experiment-core and experiment-claude from 0.2.0 to 0.3.0 * Restore agent-bench in BOM after successful 0.3.0 release * Update BOM to latest released versions across all projects * Prepare for next development iteration 1.0.13-SNAPSHOT ## 1.0.12 (2026-05-16) * Migrate BOM from org.springaicommunity to io.github.markpollack * Update .gitignore * Prepare for next development iteration 1.0.12-SNAPSHOT ## 1.0.11 (2026-04-15) * Bump agent-workflow modules 0.4.0 → 0.5.0 * Prepare for next development iteration 1.0.11-SNAPSHOT ## 1.0.10 (2026-04-14) * Update agent-workflow to 0.4.0 (annotation model release) ## 1.0.9 (2026-04-14) * Bump acp-spring-boot-autoconfigure/starter to 0.11.0 * Update agent-client to 0.12.2, release BOM 1.0.8, bump to 1.0.9-SNAPSHOT ## 1.0.7 (2026-04-10) * Bump agent-client to 0.12.1 (codex CLI fix), BOM 1.0.7 * Bump agent-client to 0.12.0, add qwen-code modules * Bump agent-hooks to 0.6.2, add claude + gemini modules ## 1.0.6 (2026-04-10) * Bump ACP SDK from 0.9.0 to 0.10.0 ## 1.0.5 (2026-04-09) * Add agent-hooks 0.5.0, bump journal 1.0.0, agent-utils 0.7.0, testing-skills 1.0.0 * Prepare for next development iteration 1.0.5-SNAPSHOT ## 1.0.4 (2026-04-02) * Dependency/version maintenance release. ## 1.0.3 (2026-04-02) * Update agent-bench to released 0.2.1, fix SNAPSHOT blocking release * Reset dev version to 1.0.4-SNAPSHOT * Upgrade agent-workflow modules to 0.2.0 * Add agent-bench-core and agent-bench-agents 0.2.0-SNAPSHOT to BOM ## 1.0.2 (2026-03-30) * Fix central-publishing-maven-plugin version (0.10.0 not 0.11.0) * Update agent-client artifacts to 0.11.0 with renamed artifactIds ## 1.0.1 (2026-03-30) * Add agent-client starters, advisor, and launcher to BOM ## 1.0.0 (2026-03-30) * Initial release. # What's New Source: https://lab.pollack.ai/docs/claude-agent-sdk/whats-new Release notes for Claude Agent SDK ## 1.5.0 (2026-08-19) A security, packaging and behaviour-correction release. No public API changed; every 1.4.0 program compiles against 1.5.0 unmodified. **Consumer dependency floors.** If you depend on `claude-code-sdk` without importing a BOM, 1.4.0 resolves Jackson 2.21.2 and Jackson 3.0.3 — 23 known vulnerabilities, 8 of them HIGH. The project's own build was clean, which is why this went unnoticed: POM flattening strips ``, so the parent's Jackson BOM imports never travelled to consumers. 1.5.0 declares the floors — Jackson 2.21.6 and Jackson 3.1.6 — directly on the published module, and the same no-BOM consumer resolves zero findings. Upgrading is the fix; nothing makes 1.4.0 safe retroactively. **`connect()` no longer sends a synthetic `"Hello"`.** Both the sync and the async client used to substitute the literal string `"Hello"` when `connect()` was called with no initial prompt, and send it as a user message — contradicting the documented contract ("connects to the Claude CLI without an initial prompt") and billing a model turn the caller never requested. `connect()` now starts and initialises the session and writes nothing to the CLI, matching the official Python SDK. `connect(String)` is unchanged and still sends exactly the prompt you pass; if you relied on the old behaviour, call `connect("Hello")` explicitly. **Packaging.** The binary, sources and Javadoc archives now embed the Apache 2.0 licence text. A CycloneDX 1.6 SBOM is published for the first time. The published POM resolves from Maven Central only. **Hygiene.** A hard-coded developer path shipped in `ClaudeCliDiscovery` since the first release; it is gone. Prompt text and the full CLI command line are no longer written to INFO logs. ## 1.4.0 (2026-06-15) * Bump MCP SDK 0.15.0 -> 2.0.0 (clears CVE-2026-35568; mcp-core 2.0.0) ## 1.3.0 (2026-06-06) * Allow manual CI runs via workflow\_dispatch * Step 2.1-2.2: Retain raw JSON on RegularMessage; wire fixtures for CLI 2.1.162 ## 1.2.0 (2026-05-29) * Fix ClaudeSyncClient hang when spawned after prior session hits max turns ## 1.1.0 (2026-05-15) * Add javadoc plugin to release profile, fix stale references * Fix 2 failing ITs: CLI flag parity and robust streaming NPE * Add distributionManagement for snapshot publishing * Skip tests on snapshot publish — CI handles test validation * Add agent-clis and API key secrets to CI and snapshot workflows * Migrate to markpollack org: package rename, standalone POM, build-tools workflows * Fix README install section: remove stale snapshot repo config * Update README: released version 1.0.0 ## 1.0.0 (2026-03-05) * Initial release. # Agent Experiment API Reference Source: https://lab.pollack.ai/docs/experiment-driver/api-reference Configuration, dataset format, invoker contract, and result model ## ExperimentConfig ```java theme={null} ExperimentConfig.builder() .experimentName("name") // Required: experiment identifier .datasetDir(Path.of("dataset")) // Required: dataset directory .model("sonnet") // Required: LLM model .promptTemplate("{{task}}") // Required: prompt with placeholders .perItemTimeout(Duration.ofMinutes(2)) // Required: per-item timeout .itemFilter(ItemFilter.bucket("A")) // Optional: filter items .knowledgeBaseDir(Path.of("kb")) // Optional: KB root .outputDir(Path.of("results")) // Optional: persist workspaces .experimentTimeout(Duration.ofHours(1)) // Optional: overall timeout .metadata(Map.of("key", "value")) // Optional: arbitrary metadata .baselineId("abc123") // Optional: comparison baseline .build(); ``` ## AgentInvoker Single-method interface — implement this to plug in any agent: ```java theme={null} public interface AgentInvoker { InvocationResult invoke(InvocationContext context) throws AgentInvocationException; } ``` **Contract**: * Blocking: returns when agent completes, times out, or fails * Thread-safe: callable from multiple threads * NOT responsible for: timeout enforcement, workspace setup, result tracking ### Template Invokers The [agent-experiment-template](https://github.com/markpollack/agent-experiment-template) provides a hierarchy of ready-made invokers. Choose based on your orchestration needs: | Class | Extends | Use When | | ------------------------------ | ------------------------------ | ----------------------------------------------------------------------- | | `TemplateAgentInvoker` | `AbstractTemplateAgentInvoker` | Single-phase agent via `AgentClient` (rename to `{Domain}AgentInvoker`) | | `TwoPhaseTemplateAgentInvoker` | `AbstractTemplateAgentInvoker` | Two-phase explore + act via `ClaudeSyncClient` | | `WorkflowAgentInvoker` | `AbstractTemplateAgentInvoker` | Single-step workflow — simplest workflow entry point | | `WorkflowInvoker` | `AgentInvoker` (direct) | Multi-step typed workflow with cost tracking | **`AbstractTemplateAgentInvoker`** provides: * Pre/post invoke hooks (`preInvoke`, `postInvoke`) * Knowledge file injection into workspaces * Phase capture collection **`WorkflowAgentInvoker`** wraps a single `ClaudeStep` in a `Workflow` with journal integration wired automatically — step events are recorded as `WorkflowStepEvent` entries via `WorkflowJournal`. The experiment name is pulled from `context.metadata("experimentId")`. **`WorkflowInvoker`** is the base for multi-step workflows with typed state. Journal and cost tracking are built in. Subclasses implement three methods: ```java theme={null} public class MyWorkflow extends WorkflowInvoker { @Override protected String workflowName() { return "my-experiment"; } @Override protected Workflow buildWorkflow( InvocationContext ctx, WorkflowExecutor executor) { return Workflow.define(workflowName()) .withExecutor(executor) // journal-backed .step(analyzeStep) .step(fixStep) .build(); } @Override protected MyState buildInitialState(InvocationContext ctx) { return new MyState(ctx.workspacePath()); } } ``` ## InvocationContext What the runner passes to your agent: | Field | Type | Description | | --------------- | ---------- | ----------------------------------------- | | `workspacePath` | `Path` | Directory where agent operates | | `prompt` | `String` | Fully constructed prompt | | `systemPrompt` | `String` | Optional additional system instructions | | `model` | `String` | Model identifier | | `timeout` | `Duration` | Timeout hint | | `metadata` | `Map` | Pass-through (experimentId, itemId, etc.) | | `runDir` | `Path` | Optional directory for trace artifacts | ## InvocationResult What your agent returns: ```java theme={null} // Success InvocationResult.completed(phases, inputTokens, outputTokens, thinkingTokens, totalCostUsd, durationMs, sessionId, metadata); // Timeout InvocationResult.timeout(durationMs, metadata, errorMessage); // Error InvocationResult.error(errorMessage, metadata); ``` | Field | Type | Description | | -------------- | ---------------- | ------------------------------- | | `success` | `boolean` | Agent completed without error | | `status` | `TerminalStatus` | `COMPLETED`, `ERROR`, `TIMEOUT` | | `inputTokens` | `int` | Total input tokens consumed | | `outputTokens` | `int` | Total output tokens produced | | `totalCostUsd` | `double` | Estimated cost | | `durationMs` | `long` | Wall-clock execution time | ## ExecutionDetail Marker interface that decouples shared experiment infrastructure (ComparisonEngine, ResultStore, VerdictExtractor) from domain-specific per-item execution details. ```java theme={null} public interface ExecutionDetail { // Marker — shared infrastructure stores but never interprets } ``` | Implementation | Used By | Contains | | ---------------------- | ----------------- | ------------------------------------------------- | | `InvocationResult` | `AgentExperiment` | Agent invocation output, tokens, cost, phases | | `JudgeExecutionDetail` | `JudgeExperiment` | Candidate judgment, expected label, scorer result | `ItemResult.executionDetail()` returns `@Nullable ExecutionDetail`. Consumers use `instanceof` pattern matching to access domain-specific fields: ```java theme={null} if (item.executionDetail() instanceof InvocationResult inv) { System.out.println("Cost: $" + inv.totalCostUsd()); } ``` ## Dataset Format ### dataset.json ```json theme={null} { "schemaVersion": 1, "name": "dataset-name", "version": "1.0.0", "description": "What this dataset tests", "items": [ { "id": "ITEM-001", "slug": "short-description", "path": "items/ITEM-001", "bucket": "A", "taskType": "task-type", "status": "active" } ] } ``` ### item.json ```json theme={null} { "schemaVersion": 1, "id": "ITEM-001", "slug": "short-description", "developerTask": "Natural language task description", "taskType": "task-type", "bucket": "A", "noChange": false, "knowledgeRefs": ["path/to/kb-entry.md"], "tags": ["tag1", "tag2"], "status": "active" } ``` ### Directory layout ``` dataset/ ├── dataset.json └── items/ └── ITEM-001/ ├── item.json ├── before/ # Starting state │ └── src/... └── reference/ # Correct result └── src/... ``` ## ItemFilter ```java theme={null} ItemFilter.all() // No filtering ItemFilter.bucket("A") // Single bucket ItemFilter.tags("rename", "simple") // By tags ItemFilter.id("ITEM-001") // Single item ``` ## ResultStore | Implementation | Use case | | ----------------------------- | ----------------------------- | | `FileSystemResultStore(path)` | Production — persists to disk | | `InMemoryResultStore()` | Testing — HashMap-backed | Both implement: ```java theme={null} void save(ExperimentResult result); Optional load(String id); List listByName(String experimentName); Optional mostRecent(String experimentName); ``` ## ExperimentResult | Method | Type | Description | | ------------------ | ------------------ | ---------------------------- | | `experimentId()` | `String` | Unique run ID | | `experimentName()` | `String` | Experiment name from config | | `items()` | `List` | Per-item results | | `passCount()` | `int` | Items that passed all judges | | `failCount()` | `int` | Items that failed | | `passRate()` | `double` | Pass count / total (0.0–1.0) | *** ## Re-Evaluation Re-evaluate stored experiment results with a different jury without re-invoking the system under test. ### ReEvaluationContextFactory Functional interface that reconstructs a `JudgmentContext` from a stored `ItemResult`: ```java theme={null} @FunctionalInterface public interface ReEvaluationContextFactory { Optional create(ItemResult item); } ``` Returns `Optional.empty()` when re-evaluation is not possible (failed item, missing execution detail, workspace not preserved). ### AgentReEvaluationContextFactory Default implementation for agent experiment results. Pattern-matches on `InvocationResult` to reconstruct the context: ```java theme={null} ReEvaluationContextFactory factory = AgentReEvaluationContextFactory.defaults(); ``` Maps `TerminalStatus` to `ExecutionStatus` (`COMPLETED` → `SUCCESS`, `TIMEOUT` → `TIMEOUT`, `ERROR` → `FAILED`). Preserves original `costUsd` and `totalTokens`. ### ReEvaluator Orchestrates post-hoc re-scoring of stored experiment results: ```java theme={null} ReEvaluator reEvaluator = ReEvaluator.builder() .contextFactory(AgentReEvaluationContextFactory.defaults()) .resultStore(store) .build(); ExperimentResult reScored = reEvaluator.reEvaluate(originalResult, newJury); ``` | Method | Description | | --------------------------------------- | ---------------------------------------------------------- | | `reEvaluate(ExperimentResult, Jury)` | Re-score a loaded result with a new jury | | `reEvaluate(String experimentId, Jury)` | Load by ID, then re-score | | `agentDefaults(ResultStore)` | Convenience factory with `AgentReEvaluationContextFactory` | Re-evaluated results carry metadata: `reEvaluated=true`, `systemReinvoked=false`, `originalCostUsd`, `reEvaluationJury`, `originalTimestamp`. Skipped items carry `reEvaluationSkipped=true` with a reason. *** ## Judge Experiment Run a judge as the system under test against a labeled dataset to measure agreement rate. ### JudgeScorer Functional interface that scores a candidate judge's `Judgment` against the expected label: ```java theme={null} @FunctionalInterface public interface JudgeScorer { JudgeScorerResult score(JudgeScoringInput input); } ``` ### JudgeScoringInput ```java theme={null} public record JudgeScoringInput( DatasetItem item, // dataset item (for item-level context) Judgment actual, // candidate judge's judgment String expectedLabel // expected label from dataset ) ``` ### JudgeScorerResult ```java theme={null} public record JudgeScorerResult( boolean match, // judge agreed with expected label double score, // normalized agreement score [0, 1] String reasoning // explanation of match/mismatch ) ``` ### JudgeScorers Built-in scoring implementations: | Factory Method | Scoring Rule | | ---------------------------- | --------------------------------------------------------------------- | | `exactVerdictMatch()` | PASS/FAIL must exactly match expected `"PASS"`/`"FAIL"` label | | `exactCategoryMatch()` | `CategoricalScore` value must match expected label (case-insensitive) | | `numericalTolerance(double)` | `NumericalScore` within tolerance of expected numeric value | ### JudgeExecutionDetail Domain evidence preserved for each item: ```java theme={null} public record JudgeExecutionDetail( RecordedJudgment candidateJudgment, String expectedLabel, JudgeScorerResult scorerResult ) implements ExecutionDetail ``` Runtime candidates still return Agent Judge 0.14 `Judgment` values. Agent Experiment converts them immediately to its normalized `RecordedJudgment` persistence boundary. Existing 0.5 / Agent Judge 0.13 result files are upgraded on read; saving them writes the normalized 0.6 format. ### JudgeExperiment Builder-based experiment runner where the system under test is a `Judge`: ```java theme={null} JudgeExperimentResult result = JudgeExperiment.builder() .name("correctness-judge-calibration") .candidate(myCorrectnessJudge) .items(labeledItems) .input(item -> buildContextFromItem(item)) .expected(item -> item.metadata().get("expectedVerdict")) .scorer(JudgeScorers.exactVerdictMatch()) .resultStore(store) .build() .run(); ``` | Builder Method | Required | Description | | ----------------------------------------------- | -------- | --------------------------------- | | `name(String)` | Yes | Experiment name | | `candidate(Judge)` | Yes | Judge to evaluate | | `items(List)` | Yes | Labeled dataset items | | `input(Function)` | Yes | Builds context from item | | `expected(Function)` | Yes | Extracts expected label from item | | `scorer(JudgeScorer)` | Yes | Scoring strategy | | `resultStore(ResultStore)` | Yes | Persistence | | `datasetVersion(String)` | No | Defaults to `"1.0.0"` | Takes `List` directly — judge datasets do not require filesystem loading. ### JudgeExperimentResult ```java theme={null} public record JudgeExperimentResult( ExperimentResult experimentResult, double agreementRate, List disagreements ) ``` | Method | Description | | ------------------------ | ------------------------------------------------------------------------- | | `agreementRate()` | Fraction of items where judge agreed with expected label | | `disagreements()` | Items where judge disagreed | | `from(ExperimentResult)` | Create from an `ExperimentResult` containing `JudgeExecutionDetail` items | | `asExperimentResult()` | Unwrap for `ComparisonEngine` and `ResultStore` compatibility | ### JudgeDisagreement ```java theme={null} public record JudgeDisagreement( String itemId, JudgeExecutionDetail detail ) ``` *** ## Modules ```xml theme={null} io.github.markpollack experiment-core 0.6.0 io.github.markpollack experiment-claude 0.6.0 io.github.markpollack experiment-workflow 0.6.0 ``` # Creating Experiments Source: https://lab.pollack.ai/docs/experiment-driver/creating-experiments Design datasets, define variant ladders, filter items, and analyze results ## Design Philosophy Every experiment tests a hypothesis about what makes agents better. The experiment driver makes the independent variables explicit: | Variable | How you control it | | ----------------------- | --------------------------------------------------------------------- | | **Knowledge** | `knowledgeRefs` in dataset items, `knowledgeBaseDir` in config | | **Prompt structure** | `promptTemplate` with `{{task}}` and `{{knowledgeRefs}}` placeholders | | **Model** | `model` field in config | | **Execution strategy** | Your `AgentInvoker` implementation | | **Evaluation criteria** | Your `Jury` wiring | ## Variant Ladders The most informative experiments use a **progressive variant ladder** — each variant adds one thing to the previous: | Variant | Change from previous | Tests | | ------------------------ | ---------------------------------- | ------------------------------ | | 1. Simple prompt | — (baseline) | Model's raw capability | | 2. + System prompt | Add domain instructions | Does framing help? | | 3. + Knowledge base | Add `knowledgeRefs` | Does knowledge help? | | 4. + Skills (SkillsJars) | Same content, structured packaging | Does structure help? | | 5. + SAE | Add Structured Agent Execution | Does execution structure help? | Each step isolates one variable. If variant 4 outperforms variant 3 with identical knowledge content, the structure is what matters — not just the knowledge. ## Improvement Flywheel Variant ladders can be pre-planned, but the most effective experiments use **empirically motivated variants** — each exists because the previous variant's analysis revealed a specific gap. This follows the [Improvement Flywheel](https://github.com/markpollack/agento-forge/blob/main/concepts/improvement-flywheel.md) methodology: ``` 1. RUN — Execute a variant and capture journals 2. MEASURE — Compute scores, traces, behavioral metrics 3. DIAGNOSE — Convert signals into hypotheses about causes 4. INTERVENE — Change prompt, KB, tool, workflow, or rubric 5. VERIFY — Re-run and compare deltas / check for regressions ``` ### Iteration metadata Each variant records what motivated it using `IterationMetadata`: ```java theme={null} public record IterationMetadata(@Nullable String finding, String hypothesis) {} ``` In the [agent-experiment-template](https://github.com/markpollack/agent-experiment-template), this is configured in `experiment-config.yaml`: ```yaml theme={null} variants: - name: control promptFile: v0-naive.txt iteration: finding: null hypothesis: "Establish baseline agent behavior" - name: variant-a promptFile: v1-hardened.txt iteration: finding: "v0 BUILD→FIX loop amplification 3.2" hypothesis: "Structured execution steps reduce fix loops" ``` This creates an audit trail: for every variant you can trace back to the observation that motivated it and verify whether the hypothesis held. ### Intervention levers The type of loss determines which lever to pull: | Lever | When to use | | ----------------------- | --------------------------------------------------- | | **Prompt** | Diffuse waste, no dominant failure pattern | | **Knowledge / skills** | Friction loops around a specific knowledge gap | | **Execution structure** | Loops around states that could be deterministic | | **Model** | Agent fundamentally cannot perform the task | | **Rubric / evaluation** | Judge variance, scores don't correlate with quality | ### Comparison reporting `GrowthStoryReporter` (in the template) generates a markdown comparison report across variants. It: * Shows per-judge score deltas, improvements, and regressions for each variant pair * Flags regressions with explicit warnings when any `ScoreComparison.regressions() > 0` * Includes iteration motivation (finding + hypothesis) before each variant's scores when `IterationMetadata` is present The report is written to `analysis/comparison-report.md` and provides the MEASURE output that feeds the next DIAGNOSE step. ## Dataset Design ### Item structure Each item needs: * **`developerTask`** — what you're asking the agent to do (natural language) * **`before/`** — the starting state (real source code) * **`reference/`** — the correct result (for judge comparison) * **`bucket`** — difficulty classification (A = easy, B = medium, C = hard) * **`knowledgeRefs`** — paths to relevant KB entries (relative to `knowledgeBaseDir`) ### Buckets Use buckets to stratify difficulty: | Bucket | Typical characteristics | | ------ | -------------------------------------------------------------------------- | | **A** | Single file, mechanical change, clear instructions | | **B** | Multi-file, requires understanding, some ambiguity | | **C** | Cross-cutting concern, requires domain knowledge, creative problem-solving | ### Filtering Run subsets of the dataset: ```java theme={null} // Run only bucket A items ExperimentConfig.builder() .itemFilter(ItemFilter.bucket("A")) // ... // Run items with specific tags ExperimentConfig.builder() .itemFilter(ItemFilter.tags("rename", "simple")) // ... // Run a single item by ID ExperimentConfig.builder() .itemFilter(ItemFilter.id("RENAME-001")) // ... ``` ## ExperimentConfig Reference | Field | Required | Default | Description | | ------------------- | -------- | --------- | ------------------------------------------------- | | `experimentName` | Yes | — | Experiment identifier | | `datasetDir` | Yes | — | Path to dataset directory | | `model` | Yes | — | LLM model (`sonnet`, `opus`, `haiku`, or full ID) | | `promptTemplate` | Yes | — | Template with `{{task}}` and `{{knowledgeRefs}}` | | `perItemTimeout` | Yes | — | Timeout per item invocation | | `itemFilter` | No | all items | Filter by bucket, tags, ID, status | | `knowledgeBaseDir` | No | — | KB root (for ablation tracking) | | `outputDir` | No | — | Directory for workspaces and logs | | `experimentTimeout` | No | — | Timeout for entire experiment | | `metadata` | No | — | Arbitrary key-value pairs | | `baselineId` | No | — | Reference experiment for comparison | ## Result Structure Results are persisted by `FileSystemResultStore`: ``` results/ └── rename-field-v1/ ├── index.json # Experiment history └── a1b2c3d4.json # Individual experiment result ``` Each result contains: * Experiment metadata (name, config, git version, timestamps) * Per-item results (agent output, jury verdict, tokens, cost, duration) * Aggregate statistics (pass rate, total cost, total duration) ## Cross-Run Comparison ```java theme={null} ComparisonEngine comparison = new ComparisonEngine(); ComparisonResult diff = comparison.compare(resultA, resultB); ``` The comparison engine aligns items by ID across two experiments and reports per-item and aggregate deltas. ## Related Three-tier evaluation: deterministic, structural, semantic Full config, dataset format, invoker contract # Diagnostic Reasoning Source: https://lab.pollack.ai/docs/experiment-driver/diagnostic-reasoning Classify failure gaps, generate remediation actions, and feed the improvement flywheel ## What Diagnostic Reasoning Does After an experiment runs and the jury produces verdicts, the diagnostic system answers: **why did items fail, and what should change?** It works in three stages: 1. **Gap classification** — map each failing verdict to a category (agent error, plan gap, missing knowledge, etc.) 2. **Deterministic reasoning** — apply rule-based logic to produce actionable fixes 3. **LLM fallback** — for checks the rules can't resolve, an LLM analyzes execution traces and proposes new artifacts ``` ExperimentResult (jury verdicts) │ ▼ DiagnosticAnalyzer └─ GapClassifier: verdict → DiagnosticCheck (with GapCategory) │ ▼ DiagnosticReport (per-item checks, gap distribution) │ ▼ DiagnosticReasoner ├─ DeterministicReasoner → RemediationAction[] └─ LlmDiagnosticReasoner → RemediationAction[] + RemediationProposal[] │ ▼ RemediationReport (actions, proposals, unresolved checks) ``` ## Gap Categories Every failing verdict check is classified into a gap category that identifies **where in the system** the problem lives: | Category | Meaning | Fix target | | --------------------- | --------------------------------------------------- | ------------------------- | | `AGENT_EXECUTION_GAP` | Plan was correct, agent didn't follow it | Agent prompting | | `PLAN_GENERATION_GAP` | Plan didn't cover this pattern | Planner or planning model | | `KB_GAP` | Knowledge base doesn't cover this pattern | Add KB entry | | `TOOL_GAP` | No deterministic tool handles this | Build new tool | | `ANALYSIS_GAP` | Static analysis missed a signal | Improve analysis tools | | `CRITERIA_GAP` | VERIFY criteria were redundant/ambiguous/missing | Criteria generation | | `EVALUATION_GAP` | Jury itself is wrong (false positive/negative) | Judge calibration | | `STOCHASTICITY_GAP` | Same config produces different outcomes across runs | Requires N≥3 runs | ## DiagnosticAnalyzer Entry point for analysis. Takes an `ExperimentResult` and produces a `DiagnosticReport`: ```java theme={null} DiagnosticAnalyzer analyzer = new DiagnosticAnalyzer(gapClassifier); DiagnosticReport report = analyzer.analyze(experimentResult); report.distribution().dominant(); // e.g., AGENT_EXECUTION_GAP report.items(); // per-item ItemDiagnostic list report.recommendations(); // human-readable suggestions ``` ### GapClassifier Assigns gap categories to verdict checks. The default `HeuristicGapClassifier` uses 22 judge-specific classification rules to map failures to categories based on the judge name, check content, and available analysis data. ```java theme={null} GapClassifier classifier = new HeuristicGapClassifier(); List checks = classifier.classify(verdict, analysisEnvelope, executionPlan); ``` ### DiagnosticReport | Field | Type | Description | | ----------------- | ---------------------- | -------------------------------------------- | | `experimentId` | `String` | Experiment run ID | | `items` | `List` | Per-item classified checks with dominant gap | | `distribution` | `GapDistribution` | Aggregate counts and fractions by category | | `recommendations` | `List` | Human-readable improvement suggestions | `GapDistribution.dominant()` returns the most frequent gap category — the highest-leverage fix target. ## DiagnosticReasoner Transforms a `DiagnosticReport` into actionable remediation: ```java theme={null} public interface DiagnosticReasoner { RemediationReport reason(DiagnosticReport report, ReasoningContext context); } ``` ### ReasoningContext Provides the full data menu for reasoning — analysis output, execution plan, trajectory exhaust, and file pointers: | Field | Type | Description | | ---------------- | -------------------- | --------------------------------------- | | `analysis` | `AnalysisEnvelope` | Static analysis data (from pipeline) | | `plan` | `ExecutionPlan` | Execution roadmap (from pipeline) | | `availableTools` | `Set` | Tools available to the agent | | `phases` | `List` | Agent thinking, tool calls, and results | Helper methods: `unusedTools()`, `errorToolResults()`, `toolUsesByName(String)`. ### DeterministicReasoner Rule-based reasoning with two rule categories: **Verdict rules** (fire on failing judge checks): * Pattern-match on gap category and structured analysis data * Target specific components: planner-prompt, pom-upgrader, agent-prompt, dependency-analysis **Trajectory rules** (fire on execution context regardless of judge outcomes): * Detect efficiency gaps where the agent recovered but deterministic tools could have prevented the problem * Examples: unused tools, implicit JDK dependencies, repeated build errors, format violations ```java theme={null} DeterministicReasoner reasoner = new DeterministicReasoner(); RemediationReport report = reasoner.reason(diagnosticReport, context); report.remediations(); // actionable fixes report.unresolvedChecks(); // checks the rules couldn't resolve ``` ### LlmDiagnosticReasoner Handles checks that deterministic rules can't resolve. Analyzes execution traces (thinking, tool calls, results) and produces: * **RemediationActions** — fixes with `LLM_INFERRED` confidence * **RemediationProposals** — new deterministic artifacts (rules, KB entries, tool specs) for the flywheel ```java theme={null} public interface LlmDiagnosticReasoner { LlmReasoningResult reasonUnresolved( List unresolvedChecks, ReasoningContext context); } ``` ### CompositeDiagnosticReasoner Chains deterministic and LLM reasoning: ```java theme={null} CompositeDiagnosticReasoner reasoner = new CompositeDiagnosticReasoner( new DeterministicReasoner(), llmReasoner); RemediationReport report = reasoner.reason(diagnosticReport, context); ``` 1. Deterministic layer runs first (fast, proof-based) 2. If unresolved checks remain and an LLM fallback is available, forward them to the LLM 3. Merge results into a single `RemediationReport` If deterministic reasoning resolves all checks, the LLM is never called. ## RemediationReport | Field | Type | Description | | ------------------ | --------------------------- | -------------------------------------- | | `experimentId` | `String` | Experiment run ID | | `remediations` | `List` | Actionable fixes, highest-impact first | | `proposals` | `List` | New artifacts proposed by LLM | | `unresolvedChecks` | `List` | Checks neither layer could resolve | ### RemediationAction Each action targets a specific component and carries a confidence level: | Field | Type | Description | | ------------ | ------------ | --------------------------------------------------------------------------------------------------------------------------- | | `target` | `String` | Component to fix (e.g., "pom-upgrader", "agent-prompt") | | `actionType` | `ActionType` | `ADD_RULE`, `ENHANCE_TOOL`, `IMPROVE_PROMPT`, `ADD_KB_ENTRY`, `ENHANCE_ANALYSIS`, `CALIBRATE_JUDGE`, `MANUAL_INVESTIGATION` | | `summary` | `String` | One-line description | | `detail` | `String` | Full explanation | | `confidence` | `Confidence` | `DETERMINISTIC`, `HEURISTIC`, or `LLM_INFERRED` | ### RemediationProposal LLM-discovered patterns that can become new deterministic infrastructure: | Field | Type | Description | | ------------------ | -------------- | --------------------------------------------------------------------------------------------------------------- | | `proposalType` | `ProposalType` | `NEW_REASONER_RULE`, `KB_ENTRY_DRAFT`, `TOOL_ENHANCEMENT`, `PROMPT_PATCH`, `NEW_TOOL_SPEC`, `JUDGE_CALIBRATION` | | `target` | `String` | Component the proposal targets | | `proposalMarkdown` | `String` | Full specification for review | | `confidence` | `Confidence` | Always `LLM_INFERRED` | ## The Flywheel RemediationProposals are the flywheel mechanism. When the LLM discovers a novel failure pattern: 1. It creates a `RemediationProposal` (e.g., a new deterministic reasoner rule) 2. A human reviews and applies the proposal 3. The new rule is added to `DeterministicReasoner` 4. On the next run, that pattern is resolved deterministically — faster, cheaper, and with higher confidence Over time, the LLM fallback is invoked less as more patterns move into deterministic rules. ## Cross-Run Aggregation `DiagnosticAggregator` analyzes multiple `DiagnosticReport` instances from repeated runs to detect stochasticity: ```java theme={null} DiagnosticAggregator aggregator = new DiagnosticAggregator(); AggregatedDiagnostic agg = aggregator.aggregate(List.of(report1, report2, report3)); agg.stochasticItems(); // items with different dominant gaps across runs agg.stableItems(); // items that fail consistently for the same reason agg.stabilityFraction(); // fraction of items that are stable ``` An item classified as `AGENT_EXECUTION_GAP` in one run and `PLAN_GENERATION_GAP` in another is flagged as stochastic. Stochastic items need N≥3 runs before you can draw reliable conclusions. Stable items are immediately actionable. ## Efficiency Evaluation `EfficiencyEvaluator` scores execution efficiency across four metrics: | Metric | Weight | What it measures | | ----------------- | ------ | ---------------------------------------------- | | `buildErrors` | 0.35 | How many build errors occurred before success | | `toolUtilization` | 0.25 | Fraction of available tools actually used | | `cost` | 0.20 | LLM cost relative to a configurable ceiling | | `recoveryCycles` | 0.20 | How many error-recovery loops the agent needed | ```java theme={null} EfficiencyConfig config = new EfficiencyConfig(5.0, defaultWeights, 8); EfficiencyReport report = evaluator.evaluate(result, context, config); report.compositeScore(); // weighted average [0, 1] where 1.0 = perfect report.checks(); // per-metric breakdown ``` Metrics gracefully degrade — if data for a metric is missing, the metric is omitted rather than failing. ## Behavioral Diagnostics (Markov Analysis) Gap classification and remediation operate on **judge verdicts** — the outcome layer. A complementary diagnostic lens operates on **tool-call sequences** — the behavioral layer. The [agent-experiment-template](https://github.com/markpollack/agent-experiment-template) includes Markov chain analysis scripts that reveal *how* the agent behaves, not just *what* it produces. ### Loop amplification The Markov analysis computes **loop amplification** — how many times the agent revisits a state before moving forward. High amplification indicates friction or failure loops: | Signal | Diagnosis | Intervention | | ------------------------------------ | ----------------------------------------------------------- | ------------------------------------------------------------------ | | Amplification > 2.0 on BUILD→FIX | Agent in a fix loop — build fails, fix fails, rebuild fails | Knowledge (add fix patterns) or execution structure (pre-validate) | | Amplification > 2.0 on SEARCH states | Agent searching for something it can't find | Knowledge (add the target information) | | Amplification > 2.0 on EXPLORE | Agent reading many files without progress | Prompt (clarify task decomposition) or pre-analysis script | ### Loop types Not all loops are problems. Classify before intervening: | Loop type | Pattern | Action | | -------------- | -------------------------------------- | -------------------------------- | | **Productive** | WRITE → VERIFY → FIX → VERIFY | Leave it alone | | **Friction** | SEARCH → READ → SEARCH → READ | Add knowledge or routing | | **Failure** | BUILD → FIX → BUILD → FIX (same error) | Change strategy, not retry count | | **Diagnostic** | BUILD → ERROR → READ\_LOG → FIX | Leave it alone | | **Degenerate** | EXPLORE → EXPLORE → EXPLORE | Agent is stuck — intervene | ### Interpretation output The template's `make_markov_analysis.py` writes `analysis/markov-interpretation.md` with: * Per-variant loop amplification summary with threshold-based classification * Recommended intervention lever for each high-amplification state * Suggested next variant with a hypothesis template This connects the behavioral DIAGNOSE step to the flywheel's INTERVENE step — the interpretation tells you *which lever to pull* based on measured state-transition patterns. See the [Improvement Flywheel](https://github.com/markpollack/agento-forge/blob/main/concepts/improvement-flywheel.md) for the full methodology. ## Related Analyze, plan, and execute — the three pipeline phases Three-tier evaluation: deterministic, structural, semantic # Getting Started with Agent Experiment Source: https://lab.pollack.ai/docs/experiment-driver/getting-started Run your first AI agent evaluation: dataset, agent, jury, and variant comparison ## Quick Start with the Template The fastest way to start is the [agent-experiment-template](https://github.com/markpollack/agent-experiment-template) — a pre-wired project with variant config, analysis scripts, and the improvement flywheel methodology built in: ```bash theme={null} # Clone the template git clone https://github.com/markpollack/agent-experiment-template my-experiment cd my-experiment # Run the baseline variant ./mvnw compile exec:java -Dexec.args="--variant control" # Run all variants and compare ./mvnw compile exec:java -Dexec.args="--run-all-variants" ``` The template includes `ExperimentApp` (CLI with `--variant`, `--item`, `--run-all-variants`), a pluggable `AgentInvoker`, cascaded jury, Markov analysis scripts, and `GrowthStoryReporter` for variant comparison. Customize three things: the agent invoker, domain judges, and knowledge files. If you want to wire the experiment loop yourself from scratch, follow the steps below. ## What You'll Build An experiment that evaluates an AI agent against a dataset of coding tasks, scores the results with a jury of judges, and compares variants to test whether adding knowledge improves quality. ## Prerequisites * Java 21+ * Maven (the project includes `./mvnw`) ## Concepts A collection of items, each with a task description, "before" source state, and "reference" solution Your agent — anything that takes a prompt + workspace and produces a result One or more judges that score the agent's output against the reference Orchestrates: load items → invoke agent → evaluate → persist results ## Step 1: Create a Dataset A dataset is a directory with a manifest and per-item directories: ``` my-dataset/ ├── dataset.json └── items/ └── RENAME-001/ ├── item.json ├── before/ │ └── src/main/java/com/example/Person.java └── reference/ └── src/main/java/com/example/Person.java ``` **dataset.json** — the manifest: ```json theme={null} { "schemaVersion": 1, "name": "rename-field", "version": "1.0.0", "description": "Field rename tasks", "items": [ { "id": "RENAME-001", "slug": "simple-rename", "path": "items/RENAME-001", "bucket": "A", "taskType": "rename-field", "status": "active" } ] } ``` **item.json** — per-item metadata: ```json theme={null} { "schemaVersion": 1, "id": "RENAME-001", "slug": "simple-rename", "developerTask": "Rename the field 'name' to 'fullName' in Person.java and update all references", "taskType": "rename-field", "bucket": "A", "noChange": false, "knowledgeRefs": [], "tags": ["rename", "simple"], "status": "active" } ``` The `before/` directory is the starting state. The `reference/` directory is the correct answer. The agent never sees the reference — it's used by judges for comparison. ## Step 2: Implement an AgentInvoker ### Using the template invokers (recommended) The template ships with ready-made invokers that handle journal integration, knowledge injection, and cost tracking. Pick the one that matches your orchestration: **Single-step workflow** — the simplest starting point. Wraps a `ClaudeStep` in a `Workflow` with automatic journal recording: ```java theme={null} // WorkflowAgentInvoker works out of the box — journal wired, knowledge injected. // Rename to {Domain}AgentInvoker and override hooks as needed. ``` **Multi-step workflow** — for pipelines with typed state flowing between steps: ```java theme={null} public class MyWorkflow extends WorkflowInvoker { @Override protected String workflowName() { return "my-experiment"; } @Override protected Workflow buildWorkflow( InvocationContext ctx, WorkflowExecutor executor) { return Workflow.define(workflowName()) .withExecutor(executor) // journal + cost tracking pre-wired .step(analyzeStep) .step(fixStep) .build(); } @Override protected MyState buildInitialState(InvocationContext ctx) { return new MyState(ctx.workspacePath()); } } ``` See the [API Reference](/docs/experiment-driver/api-reference#template-invokers) for the full invoker hierarchy. ### From scratch If you need full control, `AgentInvoker` is a single-method interface: ```java theme={null} public class MyAgent implements AgentInvoker { @Override public InvocationResult invoke(InvocationContext context) { // Your agent works in context.workspacePath() // using context.prompt() as the task description ProcessBuilder pb = new ProcessBuilder( "my-agent", "--workspace", context.workspacePath().toString(), "--prompt", context.prompt()); pb.directory(context.workspacePath().toFile()); Process p = pb.start(); boolean finished = p.waitFor( context.timeout().toSeconds(), TimeUnit.SECONDS); if (!finished) { p.destroyForcibly(); return InvocationResult.timeout( context.timeout().toMillis(), context.metadata(), "Timed out"); } return InvocationResult.completed( List.of(), 0, 0, 0, 0.0, System.currentTimeMillis(), null, context.metadata()); } } ``` For Claude Code, use the built-in `ClaudeSdkInvoker` from the `experiment-claude` module. ## Step 3: Wire a Jury Start with a simple deterministic judge: ```java theme={null} public class FileExistsJudge implements Judge, JudgeWithMetadata { private final String expectedFile; public FileExistsJudge(String expectedFile) { this.expectedFile = expectedFile; } @Override public Judgment judge(JudgmentContext context) { boolean exists = Files.exists( context.workspacePath().resolve(expectedFile)); return exists ? Judgment.pass("Found") : Judgment.fail("Missing: " + expectedFile); } @Override public JudgeMetadata metadata() { return new JudgeMetadata( "file_exists", "Checks that " + expectedFile + " exists", JudgeType.DETERMINISTIC); } } Jury jury = SimpleJury.builder() .judge(new FileExistsJudge("src/main/java/com/example/Person.java"), 1.0) .votingStrategy(new MajorityVotingStrategy()) .build(); ``` ## Step 4: Run the Experiment ```java theme={null} DatasetManager datasetManager = new FileSystemDatasetManager(); ResultStore resultStore = new FileSystemResultStore(Path.of("results")); ExperimentConfig config = ExperimentConfig.builder() .experimentName("rename-field-v1") .datasetDir(Path.of("my-dataset")) .model("sonnet") .promptTemplate("{{task}}") .perItemTimeout(Duration.ofMinutes(2)) .outputDir(Path.of("results")) .build(); AgentExperiment experiment = new AgentExperiment( datasetManager, jury, resultStore, config); ExperimentResult result = experiment.run(new MyAgent()); System.out.printf("Pass rate: %.0f%% (%d/%d)%n", result.passRate() * 100, result.passCount(), result.items().size()); ``` ## Step 5: Compare Variants The real power is variant comparison — same dataset, different agent configurations: ```java theme={null} // Variant A: base agent ExperimentConfig configA = ExperimentConfig.builder() .experimentName("rename-v1-base") .datasetDir(datasetDir) .model("sonnet") .promptTemplate("{{task}}") .perItemTimeout(Duration.ofMinutes(2)) .build(); ExperimentResult resultA = runner.run(baseAgent); // Variant B: agent with knowledge base ExperimentConfig configB = ExperimentConfig.builder() .experimentName("rename-v1-with-kb") .datasetDir(datasetDir) .model("sonnet") .promptTemplate("{{task}}\n\nRelevant knowledge:\n{{knowledgeRefs}}") .knowledgeBaseDir(Path.of("knowledge")) .perItemTimeout(Duration.ofMinutes(2)) .build(); ExperimentResult resultB = runner.run(kbAgent); ``` Same model. Same dataset. Does adding curated knowledge improve agent quality? That's the thesis in action. ## What's Next Dataset design, variant ladders, and filter strategies Three-tier evaluation: deterministic, structural, and semantic # Building a Jury Source: https://lab.pollack.ai/docs/experiment-driver/jury-system Three-tier cascaded evaluation: deterministic, structural, and semantic judges ## Why a Jury? A single judge gives you a single score. A jury gives you *diagnostic information* — when something fails, you know *where* in the stack it failed and *why*. The experiment driver uses a **cascaded jury** with three tiers. Each tier is more expensive than the last, and only fires if cheaper tiers don't already have a verdict. ## The Three Tiers Zero-cost, instant, binary. Checks facts that are unambiguously right or wrong. **Examples**: Does the project compile? Does `java -version` report the right version? Are all `javax.*` imports replaced with `jakarta.*`? **Cost**: Free (no LLM calls) Compares the agent's output against the reference implementation at a structural level — AST diffs, import sets, annotation changes, POM dependency trees. **Examples**: Are the same imports present? Do method signatures match? Are the right dependencies in the POM? **Cost**: Free (structural comparison, no LLM) LLM-powered evaluation for questions that can't be answered structurally. Uses criteria extracted from the execution plan to judge whether the agent's approach was sound. **Examples**: Is the error handling strategy appropriate? Does the migration preserve business logic semantics? **Cost**: LLM tokens per item ## Wiring a Simple Jury Start with a single Tier 1 judge: ```java theme={null} Jury jury = SimpleJury.builder() .judge(new BuildSuccessJudge(), 1.0) .votingStrategy(new MajorityVotingStrategy()) .build(); ``` ## Wiring a Multi-Tier Jury Add judges from each tier with weights: ```java theme={null} // Tier 1: deterministic Judge buildJudge = new BuildSuccessJudge(); Judge versionJudge = new ClassVersionJudge(65); // Java 21 = class version 65 // Tier 2: structural Judge importJudge = new ImportDiffJudge(); Judge pomJudge = new MavenPomDiffJudge(); // Tier 3: semantic (LLM-powered) Judge semanticJudge = new SemanticDiffJudge(chatModel, criteriaExtractor); Jury jury = SimpleJury.builder() .judge(buildJudge, 1.0) // Must compile .judge(versionJudge, 0.8) // Right Java version .judge(importJudge, 0.6) // Correct imports .judge(pomJudge, 0.6) // Correct dependencies .judge(semanticJudge, 0.4) // Semantically sound .votingStrategy(new MajorityVotingStrategy()) .build(); ``` Weights determine influence on the final verdict, not ordering. The cascade is implicit in judge cost — cheap judges run first. ## Writing a Custom Judge Implement `Judge` and `JudgeWithMetadata`: ```java theme={null} public class BuildSuccessJudge implements Judge, JudgeWithMetadata { @Override public Judgment judge(JudgmentContext context) { Path workspace = context.workspacePath(); // Run the build ProcessBuilder pb = new ProcessBuilder("./mvnw", "compile"); pb.directory(workspace.toFile()); Process p = pb.start(); int exitCode = p.waitFor(); boolean success = exitCode == 0; return success ? Judgment.pass("Build succeeded") : Judgment.fail("Build failed with exit code " + exitCode); } @Override public JudgeMetadata metadata() { return new JudgeMetadata( "build_success", "Verifies the project compiles after agent modifications", JudgeType.DETERMINISTIC); } } ``` ### Judge interface | Method | Returns | Description | | ------------------------ | ---------- | --------------------------- | | `judge(JudgmentContext)` | `Judgment` | Evaluate the agent's output | ### JudgmentContext provides | Field | Type | Description | | ----------------- | ------ | ------------------------------ | | `workspacePath()` | `Path` | Agent's modified workspace | | `referencePath()` | `Path` | Reference implementation | | `itemMetadata()` | `Map` | Item metadata (id, slug, tags) | ### Judgment fields | Field | Type | Description | | ----------- | ------------------- | ------------------------------------------------------------------------------------------------- | | `status` | `JudgmentStatus` | Required outcome: `PASS`, `FAIL`, `ABSTAIN`, or `ERROR` | | `score` | `Double` (optional) | Explicit normalized score in `[0.0, 1.0]`; use `effectiveScore()` for a numeric view of PASS/FAIL | | `label` | `String` (optional) | Independent classification label when the judge assigned one | | `reasoning` | `String` | Human-readable explanation | Agent Judge 0.14 keeps outcome, normalized score, and label as independent facts. A Boolean outcome uses `Judgment.pass(...)` or `Judgment.fail(...)`; it does not duplicate the same fact in a score object. `ABSTAIN` and `ERROR` contribute no effective score. ## Diagnostic Feedback After jury evaluation, the `DiagnosticAnalyzer` classifies failures into 8 gap categories: | Gap | Where it failed | | --------------- | ---------------------------------------- | | **Knowledge** | Missing or incorrect KB entry | | **Analysis** | Pre-analysis missed a pattern | | **Planning** | Agent planned the wrong approach | | **Execution** | Agent deviated from its own plan | | **Tool** | Tool limitation or misconfiguration | | **Prompt** | Ambiguous or misleading task prompt | | **Evaluation** | Judge produced a false positive/negative | | **Environment** | External factor (timeout, network, disk) | This classification feeds the [Improvement Flywheel](/methodology/improvement-flywheel) — knowledge gaps become new KB entries, tool gaps become new deterministic tools, and the loop turns. ## Related The evaluation framework behind experiment scoring Dataset design, variant ladders, configuration # Pipeline Source: https://lab.pollack.ai/docs/experiment-driver/pipeline Three-phase orchestration: analyze a project, generate a plan, then execute with enriched context ## What the Pipeline Does `PipelineAgentInvoker` wraps any `AgentInvoker` with two optional upstream phases — project analysis and plan generation — before delegating to the actual agent. Each phase enriches the execution context so the agent starts with a better understanding of the project. ``` Phase 1: Analyze Phase 2: Plan Phase 3: Execute ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ ProjectAnalyzer │───▶│ PlanGenerator │───▶│ AgentInvoker │ │ (deterministic) │ │ (may use LLM) │ │ (delegate) │ └──────────────────┘ └──────────────────┘ └──────────────────┘ optional optional required ``` All phases are optional except execution. If analysis or planning fails, the pipeline logs a warning and continues — the agent runs with whatever context is available. ## PipelineConfig ```java theme={null} PipelineConfig config = PipelineConfig.builder() .enableAnalysis(true) // Toggle analysis phase .enablePlanning(true) // Toggle planning phase .knowledgeDir(Path.of("kb")) // KB root for planning .toolPaths(Map.of("pom-upgrader", toolJar)) // Available tools .targetBootVersion("3.4.1") // Migration target .targetJavaVersion("21") // Java target .planningModel("sonnet") // Model for planning .planningTimeout(Duration.ofMinutes(5)) // Planning timeout .build(); ``` | Field | Required | Default | Description | | ------------------- | -------- | ------- | ---------------------------- | | `enableAnalysis` | No | `true` | Run the analysis phase | | `enablePlanning` | No | `true` | Run the planning phase | | `knowledgeDir` | No | — | KB root for plan generation | | `toolPaths` | No | — | Map of tool name to JAR path | | `targetBootVersion` | No | — | Target Spring Boot version | | `targetJavaVersion` | No | — | Target Java version | | `planningModel` | No | — | Model for plan generation | | `planningTimeout` | No | 5 min | Timeout for planning phase | ## Phase 1: Project Analysis `ProjectAnalyzer` performs deterministic, read-only analysis of the workspace. No AI calls, no side effects. ```java theme={null} public interface ProjectAnalyzer { AnalysisEnvelope analyze(Path workspace, AnalysisConfig config); } ``` ### PomAnalyzer The default implementation scans Maven POM files and Java source: | What it extracts | Source | | ------------------- | -------------------------------------------------------- | | Project identity | POM `` or `` | | Spring Boot version | Parent POM or dependency management | | Java version | `java.version` or `maven.compiler.source` property | | Dependencies | Direct `` (groupId:artifactId → version) | | Modules | `` for multi-module projects | | Import patterns | `javax.*` imports grouped by namespace | | Annotations | Tracked Spring/JPA annotations with file locations | | Config files | `application.properties`, `application.yml` in resources | ```java theme={null} AnalysisConfig config = AnalysisConfig.pomOnly("3.4.1", "21"); AnalysisEnvelope envelope = new PomAnalyzer().analyze(workspace, config); envelope.projectName(); // "my-service" envelope.bootVersion(); // "2.7.18" envelope.importPatterns(); // {javax.persistence → [Entity.java, Repository.java]} envelope.annotations(); // [AnnotationUsage(Entity, Entity.java, MyEntity), ...] ``` ### AnalysisEnvelope The analysis output is an immutable record: | Field | Type | Description | | ---------------- | --------------------------- | -------------------------------------- | | `projectName` | `String` | Project name | | `buildTool` | `String` | Build tool identifier (e.g., "maven") | | `bootVersion` | `String` | Current Spring Boot version | | `javaVersion` | `String` | Current Java version | | `dependencies` | `Map` | groupId:artifactId → version | | `importPatterns` | `Map>` | Namespace → files using it | | `annotations` | `List` | Annotation, file, class name | | `configFiles` | `List` | Config files found in resources | | `modules` | `List` | Module names for multi-module projects | | `metadata` | `Map` | Includes `analysisDurationMs` | ### AnalysisStrategy | Strategy | Implementation | Description | | ---------- | -------------- | --------------------------------------------------------------- | | `POM_ONLY` | `PomAnalyzer` | POM parsing + Java file scanning (current) | | `FULL` | — | Reserved for SCIP + ASM bytecode analysis (not yet implemented) | ## Phase 2: Plan Generation `PlanGenerator` takes the analysis envelope and produces an execution roadmap. This phase may invoke an LLM. ```java theme={null} public interface PlanGenerator { ExecutionPlan generate(AnalysisEnvelope analysis, PlanConfig config); } ``` ### ClaudePlanGenerator The default implementation runs a two-phase Claude conversation: 1. **Explore** — Claude reads the knowledge store and summarizes applicable patterns 2. **Plan** — Claude generates a Forge-style roadmap with explicit tool commands, using the analysis envelope and explore output The generator extracts tool recommendations from the roadmap and tracks which KB files were accessed during planning. ### ExecutionPlan | Field | Type | Description | | --------------------- | -------------- | ----------------------------------------------------- | | `roadmapMarkdown` | `String` | Forge-style checklist (stages, RUN/VERIFY directives) | | `toolRecommendations` | `List` | Tool names extracted from roadmap | | `kbFilesRead` | `List` | KB files accessed during planning | | `inputTokens` | `int` | Planning input tokens | | `outputTokens` | `int` | Planning output tokens | | `thinkingTokens` | `int` | Planning thinking tokens | | `costUsd` | `double` | Planning cost | | `durationMs` | `long` | Planning wall-clock duration | | `planningSessionId` | `String` | Claude session ID (nullable) | ## Phase 3: Context Enrichment and Execution Before calling the delegate invoker, the pipeline enriches the prompt with analysis and planning output: * **Execution Roadmap** — the full roadmap markdown, prepended to the prompt * **Available Tools** — `java -jar` commands for each tool in `toolPaths` * **Analysis Summary** — formatted project details (Boot version, Java version, key dependencies) The delegate `AgentInvoker` receives this enriched `InvocationContext` and runs normally. The pipeline merges planning-phase metrics (tokens, cost, duration) into the final `InvocationResult`. ## Wiring a Pipeline ```java theme={null} ProjectAnalyzer analyzer = new PomAnalyzer(); PlanGenerator planner = new ClaudePlanGenerator(claudeClient); PipelineConfig config = PipelineConfig.builder() .knowledgeDir(Path.of("kb")) .toolPaths(Map.of("pom-upgrader", Path.of("tools/pom-upgrader.jar"))) .targetBootVersion("3.4.1") .planningModel("sonnet") .build(); AgentInvoker pipeline = new PipelineAgentInvoker( analyzer, planner, config, delegateInvoker); // Use like any AgentInvoker ExperimentResult result = experiment.run(pipeline); ``` Because `PipelineAgentInvoker` implements `AgentInvoker`, it plugs directly into `AgentExperiment` — existing experiment code works unchanged. ## Graceful Degradation | Failure | Behavior | | ----------------------------------------- | ------------------------------------------------------------------- | | Analysis throws `AnalysisException` | Logged as warning, planning and execution continue without analysis | | Planning throws `PlanGenerationException` | Logged as warning, execution continues with original context | | Execution fails | Propagated — this is the only phase whose failures are surfaced | ## Related Diagnose why experiments fail and generate remediation actions Group variant results and track sweep progress # Sessions and Sweeps Source: https://lab.pollack.ai/docs/experiment-driver/sessions-and-sweeps Group variant results into sessions and coordinate multi-session sweeps ## Why Sessions and Sweeps A single experiment run produces one `ExperimentResult`. When you run multiple variants — a baseline, a prompt tweak, a knowledge-base variant — you need a way to group those results and track which variants have completed. That's what sessions and sweeps provide. | Concept | What it groups | Question it answers | | -------------- | ------------------------------------------ | ------------------------------------------ | | **RunSession** | Variant results from one multi-variant run | "What happened in this run?" | | **Sweep** | Sessions across multiple runs | "Have all expected variants been covered?" | ## Hierarchy ``` Experiment (name) ├── RunSession ("full-suite-2026-03-03") │ ├── VariantEntry ("control") │ ├── VariantEntry ("variant-a") │ └── metadata │ └── Sweep ("stage5-full") ├── Expected variants: [control, variant-a, variant-b] ├── Resolutions: which session resolved each variant └── Session history (append-only audit trail) ``` Sessions group variant results from a single run. Sweeps coordinate across sessions, tracking which of the expected variants have been resolved and by which session. ## Running with Sessions To use sessions, create an `ActiveSession` and pass it to `AgentExperiment.run()`: ```java theme={null} ActiveSession active = new ActiveSession( "full-suite-2026-03-03", // session name "rename-field-v1", // experiment name "variant-a"); // variant being executed ExperimentResult result = experiment.run(agentInvoker, active); ``` When an `ActiveSession` is provided: * Traces and workspaces are written under the session directory * The result is saved to both `ResultStore` (as before) and `SessionStore` * Without an `ActiveSession`, the experiment behaves exactly as before — full backward compatibility ## SessionStore `SessionStore` persists and retrieves sessions: ```java theme={null} // Create a session RunSession session = sessionStore.createSession( "full-suite-2026-03-03", "rename-field-v1", Map.of("git", "abc123")); // Save a variant result sessionStore.saveVariantToSession( "full-suite-2026-03-03", "rename-field-v1", "variant-a", result); // Finalize sessionStore.finalizeSession( "full-suite-2026-03-03", "rename-field-v1", RunSessionStatus.COMPLETED); // Query Optional latest = sessionStore.mostRecentSession("rename-field-v1"); List all = sessionStore.listSessions("rename-field-v1"); ``` | Implementation | Use case | | ------------------------ | ------------------------------------------------ | | `FileSystemSessionStore` | Production — persists to disk with atomic writes | | `InMemorySessionStore` | Testing — HashMap-backed | ### Filesystem layout ``` results/ └── rename-field-v1/ └── sessions/ └── full-suite-2026-03-03/ ├── session.json ├── control.json └── variant-a.json ``` ## RunSession An immutable record representing a completed or in-progress session: | Field | Type | Description | | ---------------- | --------------------- | --------------------------------------------------- | | `sessionName` | `String` | Human-readable name (e.g., "full-suite-2026-03-03") | | `experimentName` | `String` | Experiment this session belongs to | | `status` | `RunSessionStatus` | `RUNNING`, `COMPLETED`, or `FAILED` | | `variants` | `List` | Per-variant results | | `metadata` | `Map` | Arbitrary key-value pairs | | `createdAt` | `Instant` | Session creation timestamp | | `completedAt` | `Instant` | Null while running | ### VariantEntry Each variant within a session carries summary metrics: | Field | Type | Description | | -------------- | -------- | ------------------------------- | | `variantName` | `String` | Variant identifier | | `experimentId` | `String` | Unique experiment run ID | | `resultFile` | `String` | Result file relative to session | | `passRate` | `double` | Fraction passed (0.0–1.0) | | `itemCount` | `int` | Total dataset items evaluated | | `costUsd` | `double` | Total LLM cost | | `durationMs` | `long` | Wall-clock duration | ## Sweeps A sweep declares which variants must run and tracks progress across sessions. This is useful when variants run at different times — overnight jobs, CI retries, or manual re-runs of failed variants. ```java theme={null} // Create a sweep with expected variants Sweep sweep = sweepStore.createSweep( "stage5-full", "rename-field-v1", List.of("control", "variant-a", "variant-b"), Map.of("stage", "5")); // Add sessions as they complete sweepStore.addSession("stage5-full", "rename-field-v1", "run-monday", "abc123"); sweepStore.addSession("stage5-full", "rename-field-v1", "run-tuesday", "abc123"); // Check progress Sweep updated = sweepStore.loadSweep("rename-field-v1", "stage5-full").get(); updated.missingVariants(); // variants not yet resolved updated.isComplete(); // true if all expected variants resolved updated.hasVersionMismatch(); // true if resolved variants used different git commits ``` ### Resolution model When you add a session to a sweep: 1. The sweep loads the session's variants via `SessionStore` 2. Each variant that matches an expected variant is marked as resolved 3. **Last-write-wins**: adding a newer session overwrites earlier resolutions for the same variant 4. Session variants not in the expected list are silently ignored 5. The session name is appended to `sessionHistory` (append-only audit trail) ### SweepStatus | Status | Meaning | | ----------- | ------------------------------------------ | | `RUNNING` | Created, no variants resolved yet | | `PARTIAL` | At least one variant resolved, but not all | | `COMPLETED` | All expected variants resolved | | `FAILED` | Finalized as failed | ### SweepStore ```java theme={null} // Remove a session (clears its resolutions, keeps audit trail) sweepStore.removeSession("stage5-full", "rename-field-v1", "run-monday"); // Finalize sweepStore.finalizeSweep("stage5-full", "rename-field-v1", SweepStatus.COMPLETED); // Query List all = sweepStore.listSweeps("rename-field-v1"); ``` | Implementation | Use case | | ---------------------- | -------------------------------------------------------- | | `FileSystemSweepStore` | Production — persists to disk, depends on `SessionStore` | | `InMemorySweepStore` | Testing — HashMap-backed | ## Version Mismatch Detection `Sweep.hasVersionMismatch()` returns `true` if resolved variants were run against different git commits. This catches a subtle problem: when you re-run a failed variant after a code change, the sweep now contains results from two different code versions. The mismatch flag lets you detect this and decide whether to accept the mixed results or re-run the full sweep. ## Related Dataset design, variant ladders, and filtering ExperimentConfig, AgentInvoker, InvocationContext, ResultStore # Your First Research Agent Source: https://lab.pollack.ai/docs/forge/getting-started Build a file-based research KB and teach an AI agent to navigate it — in 20 minutes You're going to build a file-based research knowledge base and teach an AI agent to navigate it. By the end, you'll ask a question about coding agents and get a grounded answer — sourced from real papers, not just the model's parametric memory. **Time**: \~20 minutes. **Result**: A working research KB with 5 papers, routing tables, and a research partner you can query. ## What You're Building This is not a chatbot. It's not a vector database. It's a structured file system that an agent reads directly — markdown files with routing tables that guide the agent to the right context. Three ideas make it work: 1. **Knowledge lives in files** — summaries, routing tables, and metadata are plain markdown in git 2. **The agent reads those files directly** — no embeddings, no vector search, no retrieval pipeline 3. **Routing tables guide the agent to the right context** — this replaces vector search for this class of problems The agent runs a simple loop: read a file, decide what to read next, synthesize an answer. It's not guessing. It reads files, follows routing tables, and composes answers from that context. ## Prerequisites * Python 3 (any recent version — no pip packages needed) * Git * [Claude Code](https://docs.anthropic.com/en/docs/claude-code/overview) * Internet access (for arXiv downloads) ## Step 1: Clone and Scaffold Clone Agento Studio and scaffold your research KB: ```bash theme={null} git clone https://github.com/markpollack/agento-studio.git ~/agento-studio cd ~/agento-studio claude ``` Once Claude Code opens, run the slash command: ``` /forge-research-kb ~/my-research-kb "How do coding agents use tools?" ``` The skill asks a few questions about your research scope, then scaffolds the project: ``` ~/my-research-kb/ ├── CLAUDE.md # Session bridge — teaches agent how to use this KB ├── plans/ │ ├── VISION.md # Research questions │ └── supporting_docs/ │ └── paper-tracker.md # Bibliography with status tracking ├── papers/ │ └── summaries/ # Per-paper structured summaries └── findings/ # Cross-cutting analysis ``` Open a new Claude Code session inside your KB: ```bash theme={null} cd ~/my-research-kb claude --add-dir ~/agento-studio ``` ## Step 2: Seed Your Paper Tracker Tell Claude Code to populate the tracker with seed papers: ``` Add these 5 seed papers to plans/supporting_docs/paper-tracker.md: - Yao et al. (2023) — ReAct (arXiv: 2210.03629) - Yang et al. (2024) — SWE-agent (arXiv: 2405.15793) - Jimenez et al. (2024) — SWE-bench (arXiv: 2310.06770) - Wang et al. (2024) — LLM Agents Survey (arXiv: 2308.11432) - Anthropic — Building Effective Agents (blog, no arXiv ID) Use the tracker's existing table format. Set all to P0, Unread. ``` Claude Code reads the tracker template, matches its column format, and adds the papers. ## Step 3: Ingest Papers — The Deterministic Layer Run the arXiv ingest script to download PDFs, metadata, and LaTeX source: ```bash theme={null} python3 ~/agento-studio/scripts/arxiv_ingest.py \ --from-tracker \ --tracker-file plans/supporting_docs/paper-tracker.md \ --papers-dir papers ``` You'll see progress as each paper downloads: ``` Resolved 4 arXiv IDs. [1/4] 2210.03629 [2/4] 2405.15793 [3/4] 2310.06770 [4/4] 2308.11432 Done. Stats: success=4 partial=0 failed=0 skipped=0 ``` This is the foundation. If the structure is inconsistent, the agent cannot navigate it reliably. The scripts enforce a predictable layout that the agent depends on. The Anthropic blog post doesn't have an arXiv ID — fetch it separately using Claude Code's WebFetch or save it manually. ## Step 4: Generate Your First Summary Ask Claude Code to read a paper and write a structured summary: ``` Read the LaTeX source for ReAct (papers/source/2210.03629/) and write a summary to papers/summaries/react-reasoning-acting.md using this format: ## Yao et al. (2023) — ReAct **arXiv**: 2210.03629 **Status**: Summarized ### Key Contribution {1-2 sentences} ### Key Findings - {finding 1} - {finding 2} ### Methodology {How they tested it} ### Limitations {What they didn't cover} ### Connections - Relates to: {other papers} ``` Claude Code reads the `.tex` files directly and produces a grounded summary. Update the paper tracker to mark it as `Summarized`. ## Quick Test — Your First Query Now ask a question: ``` What is the ReAct pattern? ``` The agent reads your summary and answers grounded in it — not just from training data. You'll see it navigate to `papers/summaries/react-reasoning-acting.md` and cite specific findings. Repeat Step 4 for the remaining papers, then continue. ## Step 5: Build Routing Tables With summaries written, create the routing layer. Create `papers/summaries/index.md`: ```markdown theme={null} # Paper Summaries | Summary | Read when... | |---------|-------------| | [react-reasoning-acting.md](react-reasoning-acting.md) | Question involves reasoning + acting, thought-action loops | | [swe-agent-interface.md](swe-agent-interface.md) | Question involves agent-computer interfaces, SWE-bench tooling | | [swe-bench-benchmark.md](swe-bench-benchmark.md) | Question involves coding benchmarks, evaluation | | [llm-agents-survey.md](llm-agents-survey.md) | Question involves agent taxonomy, broad landscape | | [anthropic-effective-agents.md](anthropic-effective-agents.md) | Question involves practical patterns, production systems | ## Not Covered - Multi-agent orchestration (single-agent tool use only) - Non-coding agent domains - Reinforcement learning approaches - Commercial platform internals (LangChain, CrewAI) ``` The `Read when...` column is the core mechanism. When the agent gets a question, it reads this table and follows the link whose description matches. **If the agent answers poorly, this table is usually the problem.** The `Not Covered` section prevents the agent from searching for content that doesn't exist. ## Step 6: Ask a Real Question Ask something that requires cross-summary reasoning: ``` How does SWE-agent's approach to tool use differ from the ReAct pattern? ``` The agent reads the routing table, identifies two relevant summaries, reads both, and synthesizes a comparison with citations from each paper. ## If It Doesn't Work When the agent gives a wrong or weak answer, the fix is always in the knowledge — not in prompts. | Symptom | Fix | | -------------- | ------------------------------------------------------------------ | | Wrong answer | Routing table `Read when...` descriptions don't match the question | | Missing detail | Summaries are too shallow — add more content | | Hallucination | `Not Covered` section is missing a topic | This system improves by editing knowledge, not tuning prompts. ## Step 7: Validate Your KB Run the health check to catch structural issues: ``` /kb-reindex ``` This checks for orphan files, broken cross-references, stale indexes, and missing "Not Covered" sections. Routing gaps are the most common cause of weak agent answers. ## What You Learned * Routing tables replace vector search for this class of problems * Knowledge improves by editing files, not tuning prompts * Structure enables agent navigation — the agent reads, decides, synthesizes ## Why This Works Instead of probabilistic retrieval: * You control exactly what the agent reads * Context selection is explicit, not fuzzy * Improvements are local — edit a file, not a system This trades automation for control — and that's the point. ## What Just Happened You built a research partner that answers questions grounded in real papers. This same pattern scales to codebases, issue trackers, and multi-agent systems. This is an example of the [Forge methodology](/projects/agento-studio) — a way to build agent-native knowledge systems incrementally. ## Next Steps * **Add more papers** — Expand the tracker, run the batch pipeline, write summaries * **Synthesize themes** — Write cross-cutting analysis in `findings/` * **Federate** — Connect this KB to other projects via `KB-FEDERATION.md` * **Explore the full methodology** — See the [Agento Studio project page](/projects/agento-studio) # Loopy CLI Reference Source: https://lab.pollack.ai/docs/loopy/cli-reference All flags, slash commands, execution modes, and configuration options ## Execution Modes | Mode | Command | Description | | --------- | ----------------- | ----------------------------------------------------- | | **TUI** | `loopy` | Interactive terminal UI with chat history and spinner | | **Print** | `loopy -p "task"` | Single task, output to stdout, exit code 0/1 | | **REPL** | `loopy --repl` | Simple readline loop for quick tasks | ## CLI Flags | Flag | Description | Default | | ------------------------ | ------------------------------------- | -------------------- | | `-d, --directory ` | Working directory | Current directory | | `-m, --model ` | Model to use | Per-provider default | | `-t, --max-turns ` | Maximum agent loop iterations | `20` | | `-p, --print ` | Single-shot print mode | — | | `--provider ` | `anthropic`, `openai`, `google-genai` | `anthropic` | | `--base-url ` | Custom API base URL (vLLM, LM Studio) | — | | `--debug` | Verbose agent activity on stderr | — | | `--repl` | REPL mode | — | | `--help` | Print usage | — | | `--version` | Print version | — | ## Slash Commands Lines starting with `/` are intercepted before reaching the agent — zero LLM tokens consumed. | Command | Description | | ----------------------------- | --------------------------------------------------------------------------------------- | | `/help` | List available commands | | `/clear` | Clear session memory | | `/quit` | Exit Loopy | | `/btw ` | Stateless side question — never added to conversation history | | `/skills` | Discover, search, install, and manage domain skills | | `/boot-new` | Scaffold a new Spring Boot project from a bundled template | | `/boot-setup` | One-time preferences wizard (groupId, Java version, database) | | `/starters` | Discover Agent Starters; suggest by pom.xml triggers | | `/boot-add` | Bootstrap domain capabilities into an existing project | | `/boot-modify` | Structural modifications (Java version, native support, CI) | | `/forge-agent --brief ` | Bootstrap an agent experiment project from YAML brief | | `/session save [name]` | Save current session | | `/session list` | List saved sessions | | `/session load ` | Restore a saved session | | `/model` | List or switch model within the active provider | | `/` | Any markdown command in `~/.claude/commands/` or `.claude/commands/`, loaded at startup | ## Agent Tools The embedded agent has access to these tools: | Tool | Description | | ----------------- | ------------------------------------- | | `Bash` | Execute shell commands | | `Read` | Read file contents | | `Write` | Create or overwrite files | | `Edit` | Targeted edits to existing files | | `Glob` | Find files by pattern | | `Grep` | Search file contents | | `ListDirectory` | List directory with optional depth | | `Skill` | Load domain skills on demand | | `Submit` | Submit final answer (ends the loop) | | `TodoWrite` | Track work items | | `Task` | Delegate to subagents | | `TaskOutput` | Retrieve background subagent results | | `AskUserQuestion` | Ask user for clarification (TUI only) | | `WebSearch` | Web search (requires `BRAVE_API_KEY`) | | `WebFetch` | Fetch and summarize web pages | ## Environment Variables | Variable | Description | | ------------------- | ------------------------------------------------- | | `ANTHROPIC_API_KEY` | Anthropic API key (required for default provider) | | `OPENAI_API_KEY` | OpenAI API key | | `GOOGLE_API_KEY` | Google Gemini API key | | `BRAVE_API_KEY` | Brave Search API key (for WebSearch/WebFetch) | | `LOOPY_DEBUG_LOG` | Path for debug log file | ## Default Models | Provider | Model | | --------- | -------------------------- | | Anthropic | `claude-sonnet-4-20250514` | | OpenAI | `gpt-4o` | | Gemini | `gemini-2.5-flash` | Override with `--model` or `.model()` in the programmatic API. # Extending Loopy Source: https://lab.pollack.ai/docs/loopy/extending Custom skills, subagents, tool profiles, listeners, and the programmatic API Loopy has five extension points, from zero-code Markdown files to full Java SPIs. Domain knowledge the agent reads on demand Markdown-defined commands from files Specialist agents for delegation New tools via Java SPI *** ## Skills Skills are the fastest way to make Loopy smarter. A skill is a Markdown file with YAML frontmatter. The agent sees skill names and descriptions upfront but only loads full content when relevant — no tokens wasted. ### Create a skill Place a `SKILL.md` in your project: ``` .claude/skills/my-conventions/SKILL.md ``` ```markdown theme={null} --- name: my-conventions description: Team coding conventions for our Spring Boot services --- # Instructions When working on this codebase: - Use constructor injection, never field injection - All REST endpoints return ProblemDetail for errors (RFC 9457) - Tests use @WebMvcTest with MockMvc, not @SpringBootTest - Entity IDs are UUIDs, never auto-increment ``` Next time you run Loopy in that directory, the agent discovers the skill automatically. ### Where skills live | Location | Path | Use case | | --------- | ------------------------------------ | --------------------------------------- | | Project | `.claude/skills/*/SKILL.md` | Team conventions, checked into the repo | | Global | `~/.claude/skills/*/SKILL.md` | Personal skills across all projects | | Classpath | `META-INF/skills/*/SKILL.md` in JARs | Published skill packages (Maven dep) | ### Install from the catalog Loopy ships with 23+ curated skills from 8 publishers: ```bash theme={null} /skills search testing /skills info systematic-debugging /skills add systematic-debugging ``` ### Publish skills as a JAR (SkillsJars) Package skills as a Maven dependency so teams get them automatically: ``` my-skills.jar └── META-INF/skills/ └── my-org/my-repo/api-design/ └── SKILL.md ``` Add the JAR to `pom.xml` and Loopy discovers it on the classpath. Skills follow the [agentskills.io](https://agentskills.io) spec — they work in 40+ agentic CLIs, not just Loopy. *** ## Slash Commands Slash commands let you define reusable prompts as Markdown files, following Claude Code's `~/.claude/commands/` convention. Type `/command-name` in the chat and the agent executes the expanded prompt. ### Create a slash command Create a `.md` file in your project or home directory: ``` .claude/commands/review.md ``` ```markdown theme={null} --- name: review description: "Code review the current diff" --- Review the staged changes (`git diff --cached`). For each file: 1. Check for bugs, security issues, and performance problems 2. Verify test coverage for new code paths 3. Flag any style violations Summarize findings as a table. $ARGUMENTS ``` ### How it works 1. **YAML front matter** — `name` and `description` fields. If no front matter, the filename becomes the name. 2. **`$ARGUMENTS` substitution** — replaced with whatever the user types after the command. `/review focus on error handling` substitutes "focus on error handling". 3. **Agent delegation** — the expanded prompt is sent to the agent as if the user typed it. ### Where commands are discovered | Location | Path | | -------- | ------------------------- | | Project | `.claude/commands/*.md` | | Global | `~/.claude/commands/*.md` | Markdown commands override built-in Java commands with the same name. *** ## Subagents Subagents are specialist agents the main agent delegates to via the `Task` tool. Define them as Markdown files — no Java required. ### Create a subagent Create `.claude/agents/test-runner.md`: ```markdown theme={null} --- name: test-runner description: Runs tests and reports pass/fail summary tools: Bash, Read --- You are a testing specialist. When invoked: 1. Run `./mvnw test` in the working directory 2. Parse output for pass/fail/skip counts 3. If tests fail, read the relevant test source files 4. Report a concise summary: what passed, what failed, and why ``` The main agent delegates testing tasks to this subagent automatically based on the `description` field. ### Frontmatter fields | Field | Required | Description | | ------------- | -------- | -------------------------------------------------------- | | `name` | Yes | Unique identifier (lowercase, hyphens) | | `description` | Yes | When to use — the main agent reads this to decide | | `tools` | No | Allowed tools (comma-separated). Inherits all if omitted | | `model` | No | `haiku`, `sonnet`, or `opus` | ### Tips * Keep `description` specific — "Runs tests and reports results" routes better than "helps with testing" * Restrict `tools` to what the subagent needs. A test runner doesn't need `Edit` * Subagents run in isolated context windows — they can't see the main conversation * Subagents cannot spawn other subagents (the `Task` tool is excluded automatically) *** ## Tool Profiles (Java SPI) Add new tools the agent can call. Implement a Java interface, package as a JAR, and Loopy discovers it at startup via `ServiceLoader`. ### Implement ToolProfileContributor ```java theme={null} public class DatabaseToolProfile implements ToolProfileContributor { @Override public String profileName() { return "database-tools"; } @Override public List tools(ToolFactoryContext ctx) { return List.of( ToolCallbacks.from(new QueryTool(ctx.workingDirectory())), ToolCallbacks.from(new SchemaTool(ctx.workingDirectory())) ); } } ``` ### Register via ServiceLoader Create `META-INF/services/io.github.markpollack.loopy.tools.ToolProfileContributor`: ``` com.example.tools.DatabaseToolProfile ``` ### What ToolFactoryContext provides | Field | Type | Description | | ------------------ | ----------- | --------------------------------------- | | `workingDirectory` | `Path` | Agent's working directory | | `chatModel` | `ChatModel` | The active LLM (for tools that need AI) | | `commandTimeout` | `Duration` | Tool execution timeout (default 120s) | | `interactive` | `boolean` | True in TUI mode, false in print/REPL | ### Built-in profiles | Profile | Description | | ---------- | -------------------------------------------------------------------- | | `dev` | Full interactive toolset (bash, file I/O, search, skills, subagents) | | `boot` | Spring Boot scaffolding tools | | `headless` | Same as `dev` minus `AskUserQuestion` (for CI/CD) | | `readonly` | Read-only: file read, grep, glob, list directory | Custom profiles load **alongside** built-in profiles, not replacing them. *** ## Listeners Observe what the agent does without changing its behavior. ### ToolCallListener Fires around every tool execution: ```java theme={null} public class CostTracker implements ToolCallListener { @Override public void onToolExecutionCompleted(String runId, int turn, AssistantMessage.ToolCall toolCall, String result, Duration duration) { log.info("Tool {} took {}ms", toolCall.name(), duration.toMillis()); } } ``` ### AgentLoopListener Fires at loop lifecycle boundaries: ```java theme={null} public class ProgressReporter implements AgentLoopListener { @Override public void onLoopCompleted(String runId, LoopState state, TerminationReason reason) { System.err.printf("Done: %s (%d turns)%n", reason, state.turns()); } } ``` Wire listeners through the `MiniAgent` builder: ```java theme={null} var agent = MiniAgent.builder() .config(config) .model(chatModel) .toolCallListener(new CostTracker()) .loopListener(new ProgressReporter()) .build(); ``` Listener methods should not throw exceptions — exceptions are logged and swallowed to avoid crashing the agent loop. *** ## Programmatic API Embed Loopy's agent in other Java applications: ```java theme={null} LoopyAgent agent = LoopyAgent.builder() .workingDirectory(Path.of("/path/to/project")) .build(); LoopyResult result = agent.run("add input validation to UserController"); ``` ### Multi-step with session memory Context is preserved across `run()` calls by default: ```java theme={null} LoopyAgent agent = LoopyAgent.builder() .workingDirectory(workspace) .maxTurns(80) .build(); agent.run("plan a refactoring of the service layer"); agent.run("now execute the plan"); // sees the previous conversation ``` ### Builder options | Method | Description | Default | | ----------------------------- | ------------------------------------- | ---------------------- | | `.model(String)` | Model ID | `claude-sonnet-4-6` | | `.workingDirectory(Path)` | Agent's working directory | *required* | | `.systemPrompt(String)` | Custom system prompt | Built-in coding prompt | | `.maxTurns(int)` | Max loop iterations | `80` | | `.costLimit(double)` | Max cost in dollars | `$5.00` | | `.sessionMemory(boolean)` | Preserve context across `run()` calls | `true` | | `.timeout(Duration)` | Overall loop timeout | `10 min` | | `.disabledTools(Set)` | Tools to exclude | none | ### Custom endpoints (vLLM, LM Studio) ```java theme={null} LoopyAgent agent = LoopyAgent.builder() .baseUrl("http://localhost:1234/v1") .apiKey("lm-studio") .model("local-model") .workingDirectory(workspace) .build(); ``` # Getting Started with Loopy Source: https://lab.pollack.ai/docs/loopy/getting-started Install, configure, and run your first agent session in under 5 minutes ## Prerequisites * **Java 21+** — check with `java -version`. Install via [SDKMAN](https://sdkman.io/): `sdk install java 21.0.9-librca` * **An API key** for at least one provider: | Provider | Environment Variable | Get a key | | ------------------- | -------------------- | ------------------------------------------------------- | | Anthropic (default) | `ANTHROPIC_API_KEY` | [console.anthropic.com](https://console.anthropic.com/) | | OpenAI | `OPENAI_API_KEY` | [platform.openai.com](https://platform.openai.com/) | | Google Gemini | `GOOGLE_API_KEY` | [aistudio.google.com](https://aistudio.google.com/) | ```bash theme={null} export ANTHROPIC_API_KEY=sk-ant-... ``` ## Install Each release publishes two jars. The one you run is **`loopy--exec.jar`** — the self-contained executable image, attached to the GitHub Release. The plain `loopy-.jar` on Maven Central is a library jar for embedding Loopy in Java code and is not runnable with `java -jar`. ```bash theme={null} curl -LO https://github.com/markpollack/loopy/releases/latest/download/loopy-0.5.0-exec.jar ``` ```bash theme={null} java -jar loopy-0.5.0-exec.jar --version ``` `--version` and `--help` work without an API key, so you can confirm the download before setting up a provider. ```bash theme={null} java -jar loopy-0.5.0-exec.jar ``` The TUI launches with a chat interface. Type a message and press Enter. ### Launcher script The launcher finds the jar, checks for Java, and applies the JVM flags Loopy expects: ```bash theme={null} curl -LO https://github.com/markpollack/loopy/releases/latest/download/loopy-0.5.0-exec.jar curl -LO https://raw.githubusercontent.com/markpollack/loopy/main/loopy chmod +x loopy ./loopy ``` Keep the script beside the jar, or set `LOOPY_JAR` to the jar's path. Windows users want `loopy.bat` from the same location. ### JBang ```bash theme={null} jbang loopy@markpollack/loopy ``` Requires Loopy 0.5.0 or newer. ### Build from source ```bash theme={null} git clone https://github.com/markpollack/loopy.git cd loopy ./mvnw package ./loopy ``` ### Embed the library ```xml theme={null} io.github.markpollack loopy 0.5.0 ``` ## Your First Session Loopy starts in **interactive TUI mode** by default — a terminal chat interface with real-time agent feedback and a spinner while the agent thinks. Try giving it a coding task: ``` > Add input validation to the User class — name must be non-blank, email must match RFC 5322 ``` The agent reads your codebase (via `CLAUDE.md` if present), plans the change, edits files, and verifies the result. You see token usage and estimated cost after each response: ``` tokens: 1234/567 | cost: $0.0089 ``` ## Three Execution Modes | Mode | Command | Use case | | ----------------- | ---------------------- | ------------------------------------------------ | | **TUI** (default) | `loopy` | Interactive development — chat, iterate, explore | | **Print** | `loopy -p "your task"` | Scripting and CI — single task, stdout output | | **REPL** | `loopy --repl` | Quick tasks — readline loop, no TUI overhead | ## Slash Commands Lines starting with `/` are intercepted before reaching the agent — no LLM tokens consumed. | Command | What it does | | ----------------- | --------------------------------------------------- | | `/help` | List all commands | | `/skills` | Discover, search, install domain skills | | `/boot-new` | Scaffold a new Spring Boot project | | `/starters` | Discover Agent Starters for your project | | `/boot-add` | Add capabilities to an existing project | | `/boot-modify` | Structural changes (Java version, CI, native image) | | `/btw ` | Side question without polluting session context | | `/session save` | Save current session for later | | `/clear` | Reset conversation memory | ## Add Project Context Create a `CLAUDE.md` file in your project root. Loopy reads it automatically and appends it to the agent's system prompt — same convention as Claude Code. ```markdown theme={null} # My Project - Spring Boot 3.4 with Java 21 - Use constructor injection, never field injection - Tests use @WebMvcTest with MockMvc - API returns ProblemDetail for errors (RFC 9457) ``` No CLI flags needed. The agent sees your conventions on every turn. ## What's Next Custom skills, subagents, tool profiles, and the programmatic API All flags, options, and configuration # What's New Source: https://lab.pollack.ai/docs/loopy/whats-new Release notes for Loopy ## 0.5.0 (2026-08-21) **The distribution is repaired, and the Spring AI stack moves to GA.** * Publish two artifacts instead of one. `loopy-0.5.0-exec.jar` is the self-contained executable, attached to the GitHub Release; `loopy-0.5.0.jar` on Maven Central is a plain library jar you can compile against. Before this, the published coordinate was a Spring Boot fat jar whose classes sit under `BOOT-INF/classes`, so nothing could depend on it — and no release had ever had a downloadable asset attached, which meant the documented download URL returned 404. * Add `loopy` and `loopy.bat` launchers that locate the jar, check for Java, and apply the JVM flags Loopy expects. `jbang loopy@markpollack/loopy` now works for the first time. * Move to Spring AI 2.0.0 GA with `spring-ai-agent-utils` 0.10.0 and `workflow-core` 0.10.0, which are a coupled set, plus Spring Boot 4.1.0 and Jackson at the suite floors. Loopy was the last member on a Spring AI milestone, which mattered because the AgentWorks BOM manages Spring AI 2.0.0. * Stop requiring a credential to read `--help` or `--version`. Both were previously answered after an API-key prompt; malformed command lines are now reported without one either. * Report the real version. `--version` was hardcoded, so every release since 0.2.0 announced itself as `0.1.0-SNAPSHOT`. * Fix a startup failure when `BRAVE_API_KEY` is set without a chat model configured, pin `icu4j` to 77.1, and declare the snapshot repository the POM had always been missing. # Migration: spring-ai-community to markpollack Source: https://lab.pollack.ai/docs/migration/spring-ai-community-to-markpollack Coordinate and repository changes for projects that moved from the spring-ai-community GitHub organization ## Why these projects moved Several agent-engineering projects have moved from the `spring-ai-community` GitHub organization to `markpollack`. These projects began alongside Spring AI work and were originally hosted in the Spring AI Community organization while their scope was still emerging. Over time, their purpose became clearer: they are general-purpose Java libraries for working with agents, evaluation, benchmarking, sandboxing, and agent execution. They do not depend on Spring AI, and they are not Spring AI extensions. For that reason, keeping them under `spring-ai-community` created the wrong expectation. Users could reasonably assume these projects were part of the Spring AI ecosystem or required Spring AI to use. Moving them to `markpollack` makes the boundary clearer: Spring AI Community remains focused on Spring AI community projects, while these libraries now live under the namespace where I maintain and release them directly. Nothing about this move reflects poorly on the Spring AI Community organization. It continues to be the right home for projects that extend or integrate with Spring AI. The projects listed below simply aren't in that category, so they've moved to where they belong. ## What moved | Project | Old Repository | New Repository | | ------------------------------ | ---------------------------------------------------- | -------------------------------------------- | | claude-agent-sdk-java | `spring-ai-community/claude-agent-sdk-java` | `markpollack/claude-agent-sdk-java` | | agent-client | `spring-ai-community/agent-client` | `markpollack/agent-client` | | agent-sandbox | `spring-ai-community/agent-sandbox` | `markpollack/agent-sandbox` | | agent-bench | `spring-ai-community/agent-bench` | `markpollack/agent-bench` | | agent-judge | `markpollack/agent-judge` (already) | `markpollack/agent-judge` | | claude-agent-sdk-java-tutorial | `spring-ai-community/claude-agent-sdk-java-tutorial` | `markpollack/claude-agent-sdk-java-tutorial` | ## Maven coordinate changes All migrated projects now publish under the `io.github.markpollack` groupId. | Project | Old GroupId | New GroupId | Current Version | | --------------------- | ------------------------------ | ----------------------- | --------------- | | claude-agent-sdk-java | `org.springaicommunity` | `io.github.markpollack` | 1.4.0 | | agent-client | `org.springaicommunity.agents` | `io.github.markpollack` | 0.23.0 | | agent-sandbox | `org.springaicommunity` | `io.github.markpollack` | 0.9.3 | | agent-bench | `org.springaicommunity` | `io.github.markpollack` | 0.4.0 | | agent-judge | `org.springaicommunity` | `io.github.markpollack` | 0.13.0 | ## What to change in your build **Before:** ```xml theme={null} org.springaicommunity agent-judge-core 0.9.1 ``` **After:** ```xml theme={null} io.github.markpollack agent-judge-core 0.13.0 ``` **Before:** ```xml theme={null} org.springaicommunity.agents agent-client-core 0.12.2 ``` **After:** ```xml theme={null} io.github.markpollack agent-client-core 0.23.0 ``` Import the BOM to avoid specifying versions for each artifact: ```xml theme={null} io.github.markpollack agentworks-bom 1.16.0 pom import ``` ## Package name changes Java packages follow the new groupId. Update your import statements: | Project | Old Package Root | New Package Root | | --------------------- | --------------------------------- | --------------------------------- | | claude-agent-sdk-java | `org.springaicommunity.claude.*` | `io.github.markpollack.claude.*` | | agent-client | `org.springaicommunity.agents.*` | `io.github.markpollack.agents.*` | | agent-sandbox | `org.springaicommunity.sandbox.*` | `io.github.markpollack.sandbox.*` | | agent-bench | `org.springaicommunity.bench.*` | `io.github.markpollack.bench.*` | | agent-judge | `org.springaicommunity.judge.*` | `io.github.markpollack.judge.*` | ## What did NOT move These projects remain under `spring-ai-community` with their existing coordinates: | Project | Repository | GroupId | | --------------------- | ------------------------------------------- | ----------------------- | | spring-ai-a2a | `spring-ai-community/spring-ai-a2a` | `org.springaicommunity` | | spring-ai-agent-utils | `spring-ai-community/spring-ai-agent-utils` | `org.springaicommunity` | | spring-testing-skills | `spring-ai-community/spring-testing-skills` | `org.springaicommunity` | These projects continue to use their existing repositories and Maven coordinates. No action is needed if you depend on them. **ACP Java SDK** (`acp-java-sdk`) belongs to the official [Agent Client Protocol](https://github.com/agentclientprotocol) community organization, not `spring-ai-community`. It publishes under the `com.agentclientprotocol` groupId and is not part of this migration. # Code Coverage v1 — Knowledge Injection Baseline Source: https://lab.pollack.ai/experiments/code-coverage-v1 9 variants testing progressive knowledge injection on Spring Boot test generation
COMPLETE Feb 2026
## Hypothesis Progressive knowledge injection — giving agents increasingly structured domain knowledge — improves JUnit test generation quality on Spring Boot projects more than model upgrades. ## Setup | Parameter | Value | | ---------------- | -------------------------------------------------------- | | **Target** | Spring Boot projects (gs-rest-service, spring-petclinic) | | **Variants** | 9 (baseline → full forge with SAE) | | **Evaluation** | Four-tier jury (T0-T3) | | **Build tool** | Maven | | **Agent engine** | [Agent Workflow](/projects/agent-workflow) | ## The 9 Variants | # | Variant | Knowledge Level | | - | --------------------- | -------------------------------------- | | 1 | Simple prompt | None | | 2 | + System prompt | Minimal guidance | | 3 | + Flat knowledge base | File-based domain knowledge | | 4 | + Skills (SkillsJars) | Structured, agent-accessible knowledge | | 5 | + Skills + SAE | Skills + Structured Agent Execution | | 6 | + Hardened prompt | Defensive instructions | | 7 | + Hardened + KB | Hardened + flat knowledge | | 8 | + Hardened + Skills | Hardened + structured knowledge | | 9 | + Forge (full stack) | Complete knowledge-directed execution | ## Key Findings 1. **Two independent axes discovered** — Knowledge injection and prompt hardening improve quality independently 2. **Model floor exists** — PetClinic achieves 92-94% coverage across all variants (the model already knows PetClinic) 3. **SAE is most efficient** — ~~70 expected steps~~ *(withdrawn, see below)*, \$2.84 per run 4. **Partial knowledge paradox** — Some knowledge without structure can *decrease* performance 5. **First Markov fingerprints** — Tool-call traces reveal distinct behavioral signatures per variant **Updated 2026-08-27 — "70 expected steps" could not be reproduced.** Re-running the v1 traces through the corrected library gives **36–52 expected steps** for the SAE arms, not 70. The variant naming in the data on disk (`control`, `variant-a`–`e`, `claude-haiku`, `claude-*-sae`) does not match the nine-variant table above either, so this figure appears to come from a data or code vintage that no longer exists. **Treat it as unverified rather than wrong** — it may well have been right when written. Two things in v1's favour, both checked: it is **not** affected by CT6, because every v1 variant ran exactly one run per item, which is the one case where grouping by item is correct. And its start-state exposure (**CT7**) is small — 2–11% on the four arms that do not begin in `EXPLORE`, against the 30–90% seen in v3 and v4. The *relative* efficiency ordering is likely sound; the absolute step count is not citable. ## Markov Analysis Agent behavior varies dramatically across variants even when final outcomes are similar. The Markov fingerprint analysis revealed: * **JAR cluster patterns** — How much time agents spend in dependency inspection * **Thrashing loops** — BUILD→TEST→EDIT cycles that indicate the agent is stuck * **Loop amplification** — Quantified via transition probability engineering (TPE) ## Resources Full traces, Markov analysis scripts, raw data Narrative walkthrough of the Markov findings # Code Coverage v2 — Skills vs Knowledge Bases Source: https://lab.pollack.ai/experiments/code-coverage-v2 7 variants on Spring PetClinic testing whether structured skills outperform flat knowledge injection
COMPLETE Mar 2026
## Hypothesis Structured skills (SkillsJars) outperform flat knowledge injection — not because they contain more knowledge, but because structure itself changes agent behavior. Pre-analysis (a mandatory exploration pass before writing code) further reduces wasted steps by front-loading understanding. ## Setup | Parameter | Value | | --------------------- | ------------------------------------------ | | **Target** | spring-petclinic (Boot 4.0.1) | | **Variants** | 7 | | **N-count** | 3 per variant (20 sessions total) | | **Evaluation** | Four-tier jury (T0-T3) | | **Model** | Claude Sonnet 4 | | **Agent engine** | [Agent Workflow](/projects/agent-workflow) | | **Starting coverage** | 0% (all tests deleted) | ## The 7 Variants Each variant adds one variable on top of the previous. This isolates the effect of each intervention. | # | Name | What it adds | | - | ------------------------------------ | ------------------------------------------------------------------- | | 1 | simple | Minimal prompt, no knowledge, no stopping condition | | 2 | hardened | Structured prompt + explicit stopping condition | | 3 | hardened+kb | Hardened + flat knowledge base (Spring test imports, JaCoCo config) | | 4 | hardened+skills | Hardened + SkillsJars (structured, modular knowledge packages) | | 5 | hardened+preanalysis | Hardened + mandatory pre-analysis pass before writing tests | | 6 | hardened+skills+preanalysis | Skills + pre-analysis together | | 7 | hardened+skills+preanalysis+plan-act | Two-phase: deep exploration then sustained action | ## Results (N=3) | Variant | Mean Steps | Mean Cost | T3 Quality | | ------------------------------------ | ---------- | --------- | ---------- | | hardened+skills+preanalysis | 75.0 | \$3.39 | 0.850 | | hardened+preanalysis | 80.3 | \$3.41 | 0.789 | | hardened+kb | 83.3 | \$3.21 | 0.847 | | hardened+skills+preanalysis+plan-act | 95.0 | \$5.11 | 0.878 | | hardened+skills | 101.7 | \$3.70 | 0.856 | | hardened | 103.0 | \$4.08 | 0.850 | | simple | 109.5 | \$3.47 | 0.783 | Mean cost and step count per variant *Cost and step count across 7 variants. Skills+preanalysis (variant 6) achieves the lowest step count without inflating cost. Plan-act (variant 7) pays \$5.11 for marginal quality gain.* ## Key Findings ### 1. Prompt hardening is the biggest quality driver simple → hardened: +0.067 quality, -6% steps. A free gain from structural discipline alone — no knowledge injection, just telling the agent when to stop and how to structure its work. ### 2. Pre-analysis drives efficiency at a quality cost hardened+preanalysis: -22% steps but quality drops to 0.789. The agent follows its pre-analysis plan too rigidly, missing edge cases it would have discovered through exploration. Same attention budget, worse allocation. ### 3. Skills fix pre-analysis's quality regression hardened+skills+preanalysis: -31% steps AND quality = 0.850 (matches hardened). Skills give the agent the right vocabulary for each step, so it doesn't waste attention discovering patterns. Best tradeoff in the experiment. ### 4. KB is a pure efficiency play on known codebases hardened+kb: -24% steps, quality flat. The knowledge base eliminates JAR inspection cycles (the agent no longer needs to discover Spring Boot 4 import changes). On novel codebases the effect should be larger. ### 5. Plan-act is high variance Highest quality ceiling (0.878) but also highest cost (\$5.11) and rework spiral risk. The two-phase approach (deep exploration then sustained writing) occasionally gets stuck in fix loops. ### 6. Markov model predicts step counts Zero mean bias in leave-one-out cross-validation despite formal rejection of the first-order assumption. The model is wrong in theory but useful in practice. **Updated 2026-08-27 — the "formal rejection" never happened.** That rejection came from `run_second_order_test`, which rendered its verdict from an uncorrected statistic while the correct one (`lr_pvalue`) sat unused in the same function — defect **CT4**. Re-measured properly, second-order dependence is **null** (`LR p = 1.00 / 0.97`): the first-order assumption is *not* rejected on this data. So the finding is simpler than it was written — the model predicts step counts, and the theoretical objection it was hedging against was an artifact. First-order adequacy is a property of a dataset, not a law, so it must be re-tested on each new corpus. Fixed in `agent-control-theory` `d18fc50`; the validator now abstains below 30 observations rather than asserting. ## Behavioral Analysis Every tool call across all 20 sessions was classified into one of 9 behavioral states using the Markov fingerprinting methodology. This reveals *how* variants differ, not just *whether* they produce different outcomes. ### Transition Probability Matrix Transition probability matrix across all variants *Each cell shows the probability of transitioning from one behavioral state to another. Darker cells = higher probability. The diagonal (self-loops) dominates — agents spend most time repeating the same type of action.* ### The JAR Cluster: Knowledge Friction The most distinctive behavioral signature was the JAR\_INSPECT cluster — the agent downloading and inspecting Spring Boot JARs to discover import paths that changed between Boot 3 and Boot 4. JAR inspection loop — the agent stuck in a discovery cycle *Without knowledge injection, the agent cycles through JAR inspection trying to discover Boot 4 import changes. This loop consumed 6–18% of all tool calls in variants without KB or skills.* JAR loop eliminated with skills *With skills or KB providing the correct imports, the JAR inspection loop disappears entirely. The agent goes straight from reading to writing.* ### Loop Amplification Expected cycles through each behavioral loop *The Markov fundamental matrix predicts how many times the agent cycles through each loop before absorbing. Skills+preanalysis roughly halves the EXPLORE loop — same attention budget, better allocation.* **Updated 2026-08-27 — the absolute cycle counts are withdrawn.** This caption read *"from 189 expected cycles to 93."* Those came from a chain that grouped by item rather than by run, and this experiment ran 2–3 runs against a single item — so every run of a variant was concatenated into one sequence, losing all but one absorption event and inflating expected visits about threefold (defect **CT6**). Recomputed with trajectories separated: **62.0 → 31.3**. **The halving is real; the magnitudes were not.** Both arms were inflated by the same factor, so the ratio survived while the numbers did not. ### Intervention Deltas How each intervention changes transition probabilities *Each panel shows the change in transition probabilities when adding one intervention. Red = increased probability, blue = decreased. Skills most visibly reduce the EXPLORE self-loop and increase WRITE→BUILD transitions.* ### Sankey Flow Comparison Sankey flow comparing simple vs hardened+skills+preanalysis *Tool-call flow from left to right. The simple variant (top) shows wide, diffuse flows through many states. The skills+preanalysis variant (bottom) is narrower and more directed — fewer detours, more time writing.* ### Behavioral Heatmaps Behavioral heatmap — simple variant *Simple variant: broad exploration, scattered writes, high JAR\_INSPECT activity. The agent discovers everything through trial and error.* Behavioral heatmap — hardened+skills+preanalysis *Skills+preanalysis: focused exploration phase, then sustained writing. JAR\_INSPECT is eliminated. The agent knows what it needs and writes it.* ## What Comes Next The follow-up experiment ([Code Coverage v3](/experiments/code-coverage-v3)) tests what happens when the agent has skills but the existing codebase demonstrates older patterns. Spoiler: the codebase wins. ## Resources Full dataset download, variant configs, raw traces Narrative walkthrough of the behavioral analysis Full quantitative results with figures and tables How to read agent behavioral traces — a casual explainer The first experiment establishing the methodology What happens when existing code contradicts skill guidance # Code Coverage v3 — The Exemplar Effect Source: https://lab.pollack.ai/experiments/code-coverage-v3 When existing tests use older patterns, skills can't override them. The codebase is the agent's primary teacher.
COMPLETE Apr 2026
## Hypothesis When existing test files use older patterns, the agent will follow those patterns even when skills explicitly teach the newer ones. The codebase is a stronger signal than knowledge injection. ## Setup | Parameter | Value | | ----------------------- | ----------------------------------------------------- | | **Target** | spring-petclinic (Boot 4.0.1) | | **Variants** | 2 | | **N-count** | 3 per variant (6 sessions total) | | **Evaluation** | Three-tier jury (T0-T2) | | **Model** | Claude Sonnet 4.6 | | **Baseline coverage** | 64.9% (stripped from full suite) | | **Existing test files** | 6 (using `mockMvc.perform()`, no `flush()`/`clear()`) | ## The 2 Variants | # | Name | What the agent has | | - | --------------- | ------------------------------------------------------------------------------------------------- | | 1 | simple | Two-line prompt. No process guidance. | | 2 | hardened-skills | Seven-step structured prompt. Explicit stopping condition. Read-existing-tests-first instruction. | Both variants have the same [Spring testing skills](https://github.com/spring-ai-community/spring-testing-skills) installed globally. ## Results (N=3) | Variant | N | Mean Cost | Mean Turns | T2 Quality | Final Coverage | | --------------- | - | --------- | ---------- | ---------- | -------------- | | simple | 3 | \$3.60 | 47.0 | 0.667 | 95.2% | | hardened-skills | 3 | \$3.46 | 46.7 | 0.667 | 92.9% | ### Per-Run Breakdown | Run | Variant | Cost | Turns | Duration | Final Cov. | T1 | T2 | | --- | --------------- | ------ | ----- | -------- | ---------- | ----- | ----- | | n1 | simple | \$4.07 | 42 | 20.3 min | 95.3% | 0.608 | 0.667 | | n1 | hardened-skills | \$3.99 | 52 | 16.9 min | 94.6% | 0.595 | 0.667 | | n2 | simple | \$3.72 | 52 | 18.0 min | 94.6% | 0.595 | 0.667 | | n2 | hardened-skills | \$2.62 | 37 | 12.5 min | 89.2% | 0.486 | 0.667 | | n3 | simple | \$3.00 | 47 | 13.0 min | 95.6% | 0.615 | 0.667 | | n3 | hardened-skills | \$3.77 | 51 | 16.4 min | 94.9% | 0.601 | 0.667 | ### T2 Quality Breakdown | T2 Criterion | Score | Passed? | | -------------------------------- | --------- | ------- | | test\_slice\_selection | 1.00 | Yes | | assertion\_quality | 0.80 | Yes | | error\_and\_edge\_case\_coverage | 0.80 | Yes | | domain\_specific\_test\_patterns | **0.30** | **No** | | coverage\_target\_selection | 0.80 | Yes | | version\_aware\_patterns | **0.30** | **No** | | **Average (T2)** | **0.667** | — | ## Behavioral Analysis Despite identical quality scores, the two variants navigate the codebase differently. | Metric | simple | hardened-skills | | ----------------------------------------------------- | --------------------------- | -------------------- | | Orientation phase | 72% of calls | 65% of calls | | First file read | `PetClinicApplication.java` | `pom.xml` | | Read order | production code first | existing tests first | | Redundant reads per run | 16.7 | 4.3 | | Avg tool calls | 84 | 61 | | ~~Expected steps (Markov)~~ *(withdrawn — see below)* | ~~260~~ | ~~164~~ | Combined state diagram showing both variants *Each arrow shows: simple value → hardened-skills value. The EXPLORE self-loop drops (less re-reading) and the WRITE→BUILD arrow rises — the agent builds sooner.* **Updated 2026-08-27 — the transition probabilities on this diagram are corrected.** The EXPLORE self-loop was given as **0.87 → 0.70**. Those came from the same chain as the withdrawn expected-steps row: it pooled three independent runs of each variant into one sequence. Recomputed with runs kept separate, the self-loop is **0.920 → 0.871**, and the dwell time it implies is **12.5 → 7.8** rather than 7.76 → 3.32. **The direction survives; the size does not.** The hardened prompt does return to EXPLORE less often, but by about a fifth as much as this diagram shows. The counted figures on this page — tool calls 84 → 61, redundant reads 16.7 → 4.3, quality 0.667, cost, coverage — are unaffected. The diagram image itself has not been regenerated. **Updated 2026-08-27 — the "Expected steps (Markov)" row is withdrawn.** It carried two defects. The chain pooled three independent runs into one sequence, so it saw one finish instead of three and inflated the number about threefold. It then read the answer from the row for a starting state neither variant actually began in — all three `simple` runs opened with a shell command, not a file read. Corrected for both, the figures are 75.7 and 54.7, which is the mean tool calls per run that this same table already reports as 84 and 61. The Markov step was restating a count. It is removed rather than restated, and at three runs a side the difference is marginal anyway (p = 0.06). The "half as many reading-loop cycles" figure (189 → 93) came from the same calculation and goes with it. Everything **counted** on this page stands and was re-verified against the stored run data on the same date: 4x fewer redundant reads, tool calls 84 → 61, quality 0.667 in both arms, cost $3.60 → $3.46, coverage 95.2% against 92.9%. *(An earlier version of this note also claimed the self-loop reading was unaffected because it is measured directly off the transition matrix. That was wrong — the matrix is built from the same pooled chain. See the correction under the diagram above.)* 4x fewer redundant reads, 27% fewer tool calls, same quality *4x fewer redundant reads. 27% fewer tool calls. Same quality. The efficiency story is invisible if you only look at the T2 score.* ## The Exemplar Effect: v2 vs v3 | | v2 (zero tests) | v3 (existing tests) | What changed | | ----------------- | --------------- | ------------------- | ------------------------------- | | Coverage (simple) | 92–94% | 89–96% | Converges either way | | T2/T3 quality | 0.783–0.878 | 0.667 | Exemplar patterns cap quality | | version\_aware | 0.70–1.00 | **0.30** | Existing files use `perform()` | | domain\_specific | 0.70–0.90 | **0.30** | Existing files lack flush/clear | ## Key Findings 1. **The codebase is the agent's primary teacher.** Skills and prompts are secondary signals. If the existing code demonstrates older patterns, the agent will reproduce them — even when it has explicit knowledge of the better approach. 2. **Quality ceilings come from exemplars, not prompts.** T2 = 0.667 across all 6 runs. Two variants, three runs each, identical quality score. The ceiling moved when the existing tests changed (v2 vs v3), not when the prompt changed. 3. **Efficiency gains are still real.** 27% fewer tool calls, 4x fewer redundant reads. Prompt hardening and skills make the agent faster — they just can't make it better when the codebase says otherwise. 4. **Fix the code, not the prompt.** The highest-leverage intervention for agent quality is updating the exemplars the agent will see. ## What Comes Next **v4: Fix the exemplar — but not by hand.** A separate "Boot best-practices upgrade" step — skill-driven, focused, run *before* the test-writing agent starts. Fix the code the agent will imitate, then let it imitate. Prediction: T2 rises to ≥0.85. ## Resources Variant configs, analysis scripts, raw traces Narrative walkthrough of the exemplar effect The previous experiment — skills vs knowledge bases with zero existing tests # Experiments Source: https://lab.pollack.ai/experiments/index Controlled studies measuring what moves the needle for AI agent reliability Every experiment in this lab follows the same pattern: **define variants, control for one variable, measure with the four-tier jury, analyze behavioral traces.** We don't just measure whether agents succeed — we measure *how* they behave on the way to success or failure, using Markov chain analysis of tool-call traces. ## Active Experiments Infrastructure vs prompts on SWE-bench Lite — does tooling beat prompt engineering at 300-task scale? ## Completed Experiments The exemplar effect — when existing tests use older patterns, skills can't override them. T2 = 0.667 across all 6 runs. Skills vs flat knowledge bases — 7 variants on Spring PetClinic. Skills+preanalysis cuts expected steps by roughly a quarter to a third with no quality loss (absolute step counts corrected 2026-08-27 — see the experiment page). Knowledge injection baseline — 9 variants, two independent axes discovered. ## Upcoming | Experiment | Question | Status | | ----------------- | ------------------------------------------------- | ------- | | Code Coverage v4 | Fix the exemplar with a separate upgrade step | Planned | | SWE-bench Results | Cross-experiment comparison on standardized tasks | Planned | *** ## Experiment Design Principles 1. **One variable per experiment** — Isolate the thing being tested 2. **Deterministic preprocessing** — Parse inputs before the LLM sees them (zero LLM cost) 3. **Cascaded evaluation** — T0→T1→T2→T3, cheap filters first 4. **Behavioral analysis** — Markov chains on tool-call traces, not just pass/fail 5. **Reproducibility** — All experiment repos are public with full trace data # Issue Classification — Infrastructure vs Prompts Source: https://lab.pollack.ai/experiments/issue-classification SWE-bench Lite: does infrastructure-level optimization beat prompt engineering?
IN PROGRESS Mar 2026
## Hypothesis Infrastructure-level optimization — knowledge bases, deterministic preprocessing, tool configuration, judge feedback loops — outperforms prompt-level optimization on SWE-bench Lite tasks. ## Setup | Parameter | Value | | ------------------ | ------------------------------------------------------------------------------------------------- | | **Target** | SWE-bench Lite (300 tasks) | | **Variants** | 5-variant ladder | | **Control** | Arize `ruleset_0.txt` (20 rules, test\_accuracy=0.40) | | **Key innovation** | `+pre-analysis` — deterministic preprocessing (parse `_pytest` imports → route KB, zero LLM cost) | | **Evaluation** | [Four-tier jury](/methodology/four-tier-jury) adapted for SWE-bench | ## The Variant Ladder | # | Variant | Approach | | - | --------------------- | --------------------------------------------- | | 1 | Baseline | No knowledge, standard prompt | | 2 | + Prompt optimization | Better system prompt, few-shot examples | | 3 | + Knowledge base | Flat file domain knowledge for Python testing | | 4 | + Pre-analysis | Deterministic import parsing → KB routing | | 5 | + Full infrastructure | Pre-analysis + skills + judge feedback loop | ## Current Status Stage 5 complete (125 tests). Stage 7 next: fix SmokeTest package-private bug, wire ClaudeSdkInvoker, run 5-variant ladder. ## What This Proves If the infrastructure variant significantly outperforms the prompt variant on SWE-bench — a well-studied benchmark with known baselines — it provides strong evidence for the [knowledge-directed execution thesis](/methodology/knowledge-directed-execution). ## Resources Full experiment code, variant configs, SWE-bench task selection # Pollack AI Lab Source: https://lab.pollack.ai/index Methodology, libraries, and experiments for building and improving agentic software
Pollack AI Lab
Build · Evaluate · Improve Agentic Software
The aim is simple: build agents, measure what they do, and understand what makes them better. The lab contains the methods, software, and experiments behind that work. By [Mark Pollack](https://pollack.ai), creator of [Spring AI](https://spring.io/projects/spring-ai). Methodology and executable toolkit for building and evolving agentic software. Six phases, with evaluation and retained learnings as first-class artifacts. Reusable infrastructure for running, observing, evaluating, and experimenting with agents on the JVM. Controlled experiments on what changes agent behavior and outcomes, with the variants and data published alongside each result. *** ## Highlights What's genuinely new and cool across the lab right now. `ManagedAgentStep` hands one step of your workflow to Claude Managed Agents — a full hosted sandbox with bash, files, and web. Same graph; swap a single step between local JVM, Temporal, or Anthropic's cloud. Tracks ACP 0.14.0 — elicitation (agents ask *you* for structured input), session fork & resume, and provider config — dropping a Java agent straight into Zed, JetBrains, and VS Code. Agent Judge verifies whether work from Spring AI, LangChain4j, Koog, or any CLI agent satisfies its goal — combining executable checks, LLM and RAG evaluation, and explicit jury policies. **Read the deep dive →** The AgentWorks BOM ships only behind a 9/9 convergence gate (plain / Boot 3.5 / Boot 4) — every consumer gets one coherent, vulnerability-free dependency set, verified in CI, not by hand. Agent Journal captures Claude *and* Gemini CLI runs into one portable trace + cost schema — one analysis layer across runtimes, no per-vendor lock-in. New in 1.5.0. *** ## Active release train The whole suite ships as one coordinated train, pinned by the [AgentWorks BOM](/projects/agentworks-bom) — currently **1.18.0**, spanning a dozen libraries all shipped within the last month. *** ## Core Projects Agent Client Protocol — build and consume agents for Zed, JetBrains, VS Code Multi-step agentic pipelines with typed context, quality gates, and portable runtimes Evaluation with deterministic, command, and LLM judges Claude, Gemini, and Codex as actively verified Spring services Behavioral trace capture for analysis and observability Benchmarking on real enterprise Java tasks *** ## Latest Experiments
Code Coverage v3 — The Exemplar Effect
When existing tests use older patterns, skills can't override them — the codebase is the agent's primary teacher. 2 variants on Spring PetClinic.
Apr 2026 · Complete · Details →
Code Coverage v2 — Skills vs Knowledge Bases
Do structured skills beat flat knowledge injection? Skills + pre-analysis cut steps 31% with no quality loss. 7 variants.
Mar 2026 · Complete · Details →
Code Coverage v1 — Knowledge Injection Baseline
9 variants testing progressive knowledge injection on Spring Boot test generation.
Feb 2026 · Complete · Details →
*** ## Where this work leads The software and the method get applied to real products. [Bud Spring](/projects/bud) is a family of agents for controlled software work, and a sibling project, Bud DDD, applies the same method to domain-driven-design review without tying it to Spring, Java, or any single framework. [Agento University](/product/agento-university) is a longer-term exploration of a visual operating environment for a fleet of agents that each own a codebase. These are examples of where the work leads, not the organizing idea of the lab. *** # Agento Forge Source: https://lab.pollack.ai/methodology/forge A six-phase methodology for building agentic software, plus the slash-command toolkit that runs it ## What is Forge? **Agento Forge is how a developer builds and evolves agentic software: a six-phase methodology, plus a toolkit of Claude Code slash commands that scaffold and maintain the artifacts each phase produces.** It answers a different question than the rest of this methodology section. The [Improvement Flywheel](/methodology/improvement-flywheel) is the *control loop* — how a working agent gets measured and steered once it exists. Forge is what happens *before and around* that loop: how the project comes into being, what artifacts it commits to, and who keeps it healthy afterwards. Forge does not re-explain the control loop. RUN → MEASURE → DIAGNOSE → INTERVENE → VERIFY belongs to the [Improvement Flywheel](/methodology/improvement-flywheel); Forge is the methodology that decides what gets built and hands the loop something worth measuring. ## The Reading Contract Forge is deliberately small at the point of use. The obligation is one sentence: > Read your project's trio — `VISION.md`, `DESIGN.md`, `ROADMAP.md` — and the template for the phase you are in. That is the whole contract. A session can execute a phase correctly having read only those four documents. Everything else in the corpus is reference — case law explaining why a rule is what it is. If acting correctly ever *requires* reading a concept page, that is treated as a defect: the rule has not been distilled into the template where the session meets it. ## The Six Phases | Phase | Name | Purpose | Output | | ----- | ------------- | ----------------------------------------- | ------------------------------------------ | | 0 | Vision | Define what to build and why | `VISION.md` | | 1 | Research | Deep investigation of the problem space | Research corpus, reference implementations | | 2 | Design | Technical specification and decisions | `DESIGN.md`, decision records | | 3 | Roadmap | Break the design into implementable steps | `ROADMAP.md` with entry/exit criteria | | 4 | Learning Loop | Iterative implementation with feedback | Working implementation + learnings | | 5 | Documentation | User-facing docs and tutorials | `docs/` | Plus a phase-review template that gates the transition between stages. Two properties matter more than the phase names: * **Evaluation is first-class.** Judges and benchmarks are not a phase-5 afterthought; they are how phase 4 knows whether it moved. * **Learnings are a primary artifact.** A phase that produced working code and no recorded learning is half-finished. ## Two Loops ``` DISCOVERY LOOP (Phases 0-2) EXECUTION PIPELINE (Phases 3-5) Iterate until stable Sequential after discovery stabilizes ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌──────────┐ │ Phase 0 │<─>│ Phase 1 │<─>│ Phase 2 │ ───> │ Phase 3 │──>│ Phase 4 │──>│ Phase 5 │ │ Vision │ │ Research │ │ Design │ │ Roadmap │ │ Learning │ │ Docs │ └──────────┘ └──────────┘ └──────────┘ └──────────┘ │ Loop │ └──────────┘ └──────────┘ <─> = Iterative refinement ──> = Sequential execution ``` The **Discovery Loop** iterates freely — research invalidates vision assumptions, design reveals knowledge gaps. You exit when vision, research, and design are consistent with each other. The **Execution Pipeline** is sequential — you commit to a roadmap, execute with feedback, and document the result. Going back to discovery from execution is allowed, but it is a decision, not a drift. The transition between them is the highest-leverage review point in the method. ## Five Variants The same six phases apply to five kinds of project. What differs is the feedback loop in phase 4: | Variant | Use when… | Phase 4 loop | Lifecycle | | -------------- | ------------------------------------------------ | --------------------------- | ------------------------- | | **Eval-Agent** | Building an autonomous agent evaluated by judges | Loss optimization | Finite — converges | | **Project** | Building a library, service, or application | QA review | Finite — builds and ships | | **Research** | Investigating a question or testing hypotheses | Vision ↔ research iteration | Finite — publishes | | **Steward** | Keeping an existing project or domain healthy | Health monitoring | **Ongoing** | | **KB** | Structuring a corpus for agent navigation | Navigation quality | **Ongoing** | The first three converge on a deliverable and complete. The last two do not — they are custodial. ## How to Start The methodology is executable. It ships as Claude Code slash commands that scaffold and maintain the artifacts, so the on-ramp is a command, not a document to imitate. `/forge-project` walks you from nothing to a vision, a design, and a roadmap, with a QA review loop between them. For research, `/forge-research` starts at forage and vision instead. For an agent with judge-based evaluation, `/forge-eval-agent` scaffolds judges and benchmarks alongside the trio. `/forge-kb` structures an existing pile of documents into a corpus an agent can navigate — routing tables, indexes, and a session bridge, validated against real questions. `/forge-research-kb` is the superset for corpora with named consumer projects. See [Knowledge Base Design](/methodology/knowledge-base-design) for what it builds and why. `/plan-to-roadmap` converts an ad-hoc plan into roadmap steps with entry and exit criteria. `/collect-status` produces a timestamped status report. `/kb-reindex` runs freshness checks across federated knowledge bases. `/prepare-handoff` ends a session with a documentation currency pass and a work order for whoever picks it up next — human or agent. `/prepare-kb-handoff` is the knowledge-base equivalent. The handoff is an artifact, not a courtesy. When the build phases are done but the project is not, `/forge-steward-repo` establishes ongoing custody. See [The Steward Pattern](#the-steward-pattern) below. Beyond the commands, the repository carries 14 document templates, 25 named concepts, 9 practice guides, and 5 project variants. The commands are the interface; the corpus is what they are an interface to. ## The Steward Pattern The variants that converge produce something finished. Most software is not finished — it has users, dependencies that move, and a knowledge base that goes stale the moment nobody is accountable for it. That is what a **Steward** is for. A Steward is an agent that is **continuously accountable** for a project or domain — a persistent custodian rather than a one-shot executor. In the cognitive-altitude hierarchy it sits at Level 1: above task execution, below strategic planning, with a horizon measured in days and weeks. | Level | Role | Horizon | | ----- | -------------- | -------------- | | 0 | Task executor | Minutes | | **1** | **Steward** | **Days–weeks** | | 2 | Strategist | Weeks–months | | 3 | Meta-architect | Months–years | It combines two roles that reinforce each other: * **Curator** — maintains the project's knowledge base: updates entries as APIs and patterns change, cross-references related topics, prunes stale content, federates to other knowledge bases. * **Developer** — executes roadmap items, runs builds and tests, flags regressions, watches upstream for breaking changes. Neither works alone. A curator without development context writes abstract documentation; a developer without curated knowledge repeats past mistakes. ### Where a steward's planning lives The current model gives each stewarded project a **separate repository, paired one-to-one with it**, holding the project's active planning authority — the same `VISION.md` / `DESIGN.md` / `ROADMAP.md` trio — plus an inbox for steward-to-steward obligations and an audit log. That separation is the point, and it is a privacy boundary rather than a filing preference: planning cannot live in a repository whose contents are world-readable. An earlier model that added stewardship *inside* the project has been retired for exactly that reason. The steward-repository procedure is **provisional by design** — revised from observed failures rather than worked around. The bootstrap is deliberately staged into separately inspectable transitions with a validator between them, because a single opaque command must not create repositories, rewrite a project, migrate authority, and send mail without intermediate validation. ## Related Knowledge + structured execution > model — what Forge is built to serve JIT Retrieval, the two KB types, and what `/forge-kb` builds The control loop Forge hands a project to Twenty minutes from a pile of papers to a research partner you can query # Four-Tier Jury Source: https://lab.pollack.ai/methodology/four-tier-jury Cascaded evaluation — deterministic first, LLM last ## The Problem LLM-as-judge is expensive and non-deterministic. Running GPT-4 evaluation on every agent output costs \$0.50-2.00 per assessment and produces variable results. ## The Solution A **cascaded jury** that filters through cheap, deterministic checks before reaching expensive LLM evaluation. Only work products that pass all lower tiers advance. ## The Four Tiers ### T0: Deterministic Checks that require no execution — regex matching, file existence, syntax validation, compilation checks. **Examples:** * Does the generated test file exist? * Does it compile? * Does it contain at least one `@Test` annotation? * Are import statements valid? **Cost:** Free. **Latency:** Milliseconds. ### T1: Command Checks that run shell commands and inspect exit codes or output patterns. **Examples:** * Does `mvn test` pass? * Does the coverage report show improvement? * Does `checkstyle` pass? **Cost:** Minimal (compute only). **Latency:** Seconds. ### T2: Golden Test Compares agent output against known-good reference outputs using structural similarity. **Examples:** * Does the generated test cover the same methods as the reference test? * Is the assertion strategy consistent with project conventions? * Does the test structure match golden examples? **Cost:** Minimal. **Latency:** Seconds. ### T3: LLM Assessment Semantic evaluation by a language model — reserved for cases that pass all lower tiers. **Examples:** * Is the test meaningful (not just asserting `true`)? * Does it test edge cases? * Is the test maintainable? **Cost:** \$0.50-2.00 per assessment. **Latency:** 5-15 seconds. ## Cascade Economics By filtering at each tier, typically only 30-40% of outputs reach T3. This reduces evaluation cost by **60-80%** while maintaining quality — because outputs that fail T0-T2 would fail T3 anyway. ## Implementation The four-tier jury is implemented in [Agent Judge](/projects/agent-judge) and used across all lab experiments. ## Role in the Growth Cycle The four-tier jury is the **MEASURE** step of the [Improvement Flywheel](/methodology/improvement-flywheel). Each tier maps to specific loss dimensions, and the cascade structure ensures efficient measurement before expensive LLM evaluation. ### Tier-to-Loss Dimension Mapping | Tier | Loss Dimension | What It Catches | | ---------------------- | ----------------------------------------- | ------------------------------------------------------- | | **T0** (deterministic) | Outcome loss (binary) | File doesn't exist, won't compile, missing annotations | | **T1** (command) | Tooling loss | Tests fail, coverage doesn't improve, style violations | | **T2** (golden test) | Behavioral loss (structural match) | Output doesn't match reference structure or conventions | | **T3** (LLM) | Outcome loss (semantic) + evaluation loss | Meaningless tests, missing edge cases, judge variance | ### Per-Criterion Tracking Track individual criterion scores, not just the aggregate. A rising aggregate can hide a regression in a specific criterion — for example, overall batch score improves +0.4 while a single criterion drops −0.3. The [Improvement Flywheel](/methodology/improvement-flywheel) requires per-criterion visibility to detect these hidden regressions. ### Regression Detection Every improvement can introduce regressions. After each intervention, verify: 1. **Did the targeted loss dimension decrease?** — The intervention worked as intended. 2. **Did any other dimension increase?** — If so, the diagnosis was incomplete — the fix addressed a symptom, not the root cause. 3. **Is the improvement stable across multiple runs?** — Distinguish signal from lucky variance. ## Applied In * [Code Coverage v1](/experiments/code-coverage-v1) — Full T0-T3 cascade on 9 variants * [Code Coverage v2](/experiments/code-coverage-v2) — Refined scoring, T3=0.933 for forge variant * [Issue Classification](/experiments/issue-classification) — Adapted for SWE-bench task evaluation ## Related The feedback loop that jury evaluation drives # Improvement Flywheel Source: https://lab.pollack.ai/methodology/improvement-flywheel Loss-driven iteration — from measured behavioral gaps to targeted interventions ## The Core Insight Every iteration should identify a measurable gap between desired agent behavior and observed agent behavior. That gap is the **loss signal**. The flywheel turns the loss signal into a diagnosis, then into a targeted intervention, then into a verification run. This is gradient-inspired, not gradient-computed. Agent systems are not differentiable, but their journals, scores, state transitions, and failure paths provide directional evidence about where the next intervention should be applied. ## The Cycle ``` 1. RUN — Execute variants and capture journals 2. MEASURE — Compute scores, traces, behavioral metrics 3. DIAGNOSE — Convert signals into hypotheses about causes 4. INTERVENE — Change prompt, KB, tool, workflow, rubric, or template 5. VERIFY — Re-run and compare deltas/regressions ``` Each iteration estimates where the system is failing, chooses the most promising improvement direction, applies an intervention, and measures whether the system moved in the intended direction. Variants are **empirically motivated, not pre-planned** — each variant exists because the previous variant's analysis revealed a specific gap. ## Phase 0: State Taxonomy Discovery For projects that use Markov analysis, the flywheel begins with state taxonomy discovery. You need a **state taxonomy** — the named states that the classifier maps tool calls to. This taxonomy is domain-specific and must be discovered empirically. Generate enough tool-call data to see the agent's natural behavior patterns. See raw tool name + target frequencies without a predefined taxonomy. Look for related tool calls that represent a coherent activity. Name the clusters. Each state should represent a distinct *kind of work*: exploring, building, fixing, verifying, searching, reading knowledge. Aim for 5–12 states. Group states into higher-level categories: productive work (WRITE, BUILD, VERIFY), friction (FIX, SEARCH), knowledge access (READ\_KB, READ\_SKILL). **What makes a good taxonomy:** States are verbs, not nouns — they describe what the agent is *doing*, not what it's *looking at*. Each state should have diagnostic value: its frequency change tells you something about agent quality. ## Loss Signal Taxonomy The loss signal is multi-dimensional. Not every dimension matters for every iteration, but the full surface is: | Loss Dimension | What It Measures | Example Signal | | -------------- | ------------------------------------ | -------------------------------------------------------------- | | **Outcome** | Task failure or low judge score | 3 of 10 benchmark cases fail | | **Behavioral** | Unnecessary exploration or loops | BUILD→FIX loop amplification 3.2 | | **Knowledge** | Repeated search or oracle calls | Repeated fallback inspection (e.g., Maven cache decompilation) | | **Tooling** | Errors reachable from multiple paths | Same exception from 4 different states | | **Evaluation** | Judge variance or malformed output | Non-JSON judge response 2/7 runs | | **Stability** | Large run-to-run variance | Quality scores range 0.28–0.72 | | **Regression** | One metric improves, another worsens | Batch score +0.4 but scheduling score −0.3 | The MEASURE step quantifies these signals. The DIAGNOSE step identifies which dimension dominates. The INTERVENE step targets that dimension specifically. ## Diagnostic Lenses Multiple analytical tools illuminate the loss signal. No single lens is the methodology — they are instruments in the measurement apparatus. | Lens | What It Reveals | Best For | | ----------------------------------------------- | -------------------------------------------------------------- | ------------------------------------------------ | | **Markov analysis** | State transition patterns, loop amplification, transition gaps | Behavioral loss — where the agent gets stuck | | **[Judge scores](/methodology/four-tier-jury)** | Per-criterion quality assessment | Outcome loss — what the agent produces | | **Reasoning/intent traces** | Intent-to-action policy, planning distribution | Knowledge loss — what the agent is searching for | | **Oracle call log** | KB gaps the agent couldn't resolve alone | Knowledge loss — what's missing from the KB | | **Cost/token accounting** | Where the budget goes | Behavioral loss — which states burn tokens | | **Run-to-run comparison** | Variance across identical inputs | Stability loss — what's nondeterministic | ## Loop Types Not all loops are problems. The DIAGNOSE step must classify the type of loop before choosing an intervention. | Loop Type | Pattern | Meaning | Action | | -------------- | -------------------------------------- | --------------------------------------------- | -------------------------------- | | **Productive** | WRITE → VERIFY → FIX → VERIFY | Expected refinement cycle | Leave it alone | | **Friction** | SEARCH → READ → SEARCH → READ | Agent lacks context or structure | Add knowledge or routing | | **Failure** | BUILD → FIX → BUILD → FIX (same error) | Agent repeats an invalid strategy | Change strategy, not retry count | | **Diagnostic** | BUILD → ERROR → READ\_LOG → FIX | Agent is gathering useful failure information | Leave it alone | | **Degenerate** | EXPLORE → EXPLORE → EXPLORE | No new information is being gained | The agent is stuck — intervene | Optimizing loop amplification to zero is an anti-pattern. Some loops are productive. The goal is to eliminate *friction*, *failure*, and *degenerate* loops while preserving *productive* and *diagnostic* ones. ## Intervention Levers The type of loss determines which lever to pull. ### Lever 1: Prompt Clarify task decomposition, add stopping conditions, add execution ordering. The `simple` → `hardened` jump in code-coverage experiments produced the single largest quality gain (+0.07) with no external knowledge — just structure and an explicit stopping condition. *Pull when:* diffuse waste, no dominant failure pattern, agent doesn't know when it's done. ### Lever 2: Knowledge and Skills Add domain recipes, examples, routing hints. Targeted KB entries eliminate specific search loops without touching the prompt. In one observed experiment, a single knowledge package reduced JAR\_INSPECT from 18% to under 2% of all steps. *Pull when:* friction loops around a specific knowledge gap. ### Lever 3: Execution Structure Three sub-levers that replace exploratory LLM behavior with deterministic execution: * **Deterministic tools** — Replace states that don't require reasoning. A build script that returns structured results eliminates the BUILD/FIX reasoning loop. * **Templates and scaffolds** — Pre-generate structure or use cached known-good baselines. When the flywheel reveals the agent consistently discovers the same pattern through exploration, codify it. * **Steering** — Runtime hooks that intercept tool calls and enforce behavioral constraints. *Pull when:* loops around states that could be deterministic, agent repeatedly discovers the same answer, or agent makes predictable wrong choices. ### Lever 4: Model Pick a model that clears the capability floor — below it, nothing else helps. But above that floor, the other levers are cheaper and often more effective. *Pull when:* the agent fundamentally cannot perform the task, even with perfect knowledge and structure. ### Lever 5: Rubric and Evaluation Tighten judge criteria, add anchors with concrete examples, add per-criterion scoring. A rubric intervention doesn't change the agent — it changes the measurement, which changes what the next iteration optimizes for. *Pull when:* evaluation loss dominates (judge variance, malformed output, scores that don't correlate with actual quality). **The critical distinction:** Knowledge can't fix a reasoning gap. Steering can't fix a knowledge gap. A better model can't fix either. Diagnose which problem you have before you reach for a lever. ## The Deterministic-Over-Exploratory Principle The flywheel's purpose is to **systematically shrink the agent's exploration space**. When the measurement apparatus reveals the agent consistently discovers the same pattern through exploration, that pattern should be codified as a deterministic step. | Execution Path | Quality Range | Reliability | | ------------------------------------- | ------------- | ------------------ | | Cached templates (deterministic) | 0.70 – 0.93 | Stable across runs | | Expansion path (LLM with constraints) | 0.28 – 0.72 | Varies by run | | Raw Claude Code (pure exploration) | 0.19 – 0.63 | High variance | Every decision point the LLM doesn't have to make is a source of variance eliminated. LLM steps are reserved for genuinely creative decisions where the search space can't be pre-constrained. | Finding | Codification | | ---------------------------------------------- | -------------------------------------------- | | Agent always discovers the same file structure | Template or scaffold | | Agent always applies the same fix pattern | Recipe in `knowledge/` | | Agent always needs the same context | Structured context in the prompt | | Agent always makes the same tool-call sequence | Deterministic workflow step | | Agent's orientation thinking dominates | Pre-analysis script that front-loads context | ## Variant Progression Variants are empirically motivated. Each exists because the previous variant's analysis revealed a specific gap. ``` v0: baseline (control) → Run, measure: identify dominant loss dimension v1: address the dominant loss → Typically prompt improvement (Lever 1) — clearest signal first → Run, measure: did the loss decrease? What's the next loss? v2: address the next loss → Typically knowledge injection (Lever 2) — domain files for remaining gaps → Run, measure: repeat v3+: address remaining losses → Structural fixes (Lever 3), rubric tightening (Lever 5) → Each variant is motivated by the previous variant's measurement ``` Every variant links back to its motivating finding and hypothesis, creating an audit trail: for every variant you can trace back to the observation that motivated it and verify whether the hypothesis held. ## Verification Discipline The VERIFY step requires tracking what changed between iterations and what improved. **Per-iteration record:** ``` Iteration: v0 → v1 Change: Added structured execution steps to prompt Metrics before: batch score 0.519, BUILD→FIX amplification 3.2 Metrics after: batch score 0.926, BUILD→FIX amplification 1.1 Delta: +0.407 outcome score, −2.1 behavioral amplification Regression: none detected ``` **What to track:** * **Per-criterion scores** — Not just the aggregate. A rising aggregate can hide a regression in a specific criterion. * **Loop amplification per state** — The primary behavioral metric. Did the friction loop shrink? * **Transition probabilities** — Did the agent's navigation pattern change as expected? * **Variant-over-variant delta** — Before/after comparison for the specific change made. * **Stability** — Run the same variant multiple times to distinguish signal from variance. **Regression detection:** Every improvement can introduce regressions. Did the targeted loss decrease? Did any other dimension increase? Is the improvement stable across multiple runs? ## Anti-Patterns * **Skipping taxonomy discovery** — Jumping to Markov analysis with a generic state taxonomy * **Figures without interpretation** — Running analysis and looking at pictures without mapping findings to interventions * **Unmotivated variants** — Creating variants without a clear hypothesis from prior measurement * **Aggregate-only scoring** — Tracking only the overall batch score instead of per-criterion metrics * **Wrong lever** — Throwing knowledge at a reasoning gap, or a bigger model at a knowledge gap * **Fixing without verifying** — Making an improvement and moving on without confirming it worked * **Over-rotation on a single metric** — Optimizing loop amplification to zero removes productive loops * **Ignoring the deterministic principle** — Improving agent exploration instead of converting exploration into deterministic steps ## Evidence | Project | Iterations | Key Finding | | ------------------------------------------------------- | ------------------- | -------------------------------------------------------------------------------- | | [Code Coverage v1→v2→v3](/experiments/code-coverage-v1) | 7 variants, 20 runs | Structure beats knowledge; skills reduce waste but not quality ceiling | | bud-eval | 6 iterations | Batch 0.519→0.926 (template fix), scheduling oscillation→stable 0.741 (test fix) | ## Related The development methodology that builds what this loop then measures Cascaded evaluation — the MEASURE step IterationMetadata and variant progression in practice # Knowledge Base Design Source: https://lab.pollack.ai/methodology/knowledge-base-design How to structure domain knowledge for agent consumption — routing tables, progressive disclosure, and the Diataxis weighting ## The Problem Agents read files. The question is: which files, in what order, with what structure? A flat directory of Markdown files forces the agent to read everything or guess. A well-structured KB lets the agent navigate to exactly what it needs in 1-2 file reads. ## JIT Retrieval The approach on this page has a name: **JIT Retrieval**, also called **Explore RAG**. It is knowledge retrieval with **no vector store, no embeddings, and no indexing pipeline**. The knowledge is structured markdown in git, and the agent navigates it with the file tools it already has — glob, grep, read. There is nothing to stand up and nothing to keep in sync with the files. That sounds like a limitation. It is closer to the opposite, for five reasons: 1. **Routing tables are human-authored rerankers.** A "read when…" column encodes domain expertise about what is relevant to what. That beats statistical similarity, because it is a judgment about relevance rather than a measurement of resemblance. 2. **Negative knowledge saves more time than positive routing.** Recording what *is not* here prevents the most wasteful searches — the ones that end in nothing after reading half the corpus. 3. **Hierarchical navigation is O(log n).** Root index → domain index → file. Adding files does not lengthen the path. 4. **No infrastructure.** No embedding model, no vector store, no indexing job, no re-embedding when a document changes. A `git commit` is the entire update pipeline. 5. **It works to roughly 500 files per KB.** Beyond that, federate rather than growing a single corpus. **The quality metric is the right file in three hops or fewer.** That is the number to test against, and the reason index files stay short. Maintenance is two-agent: a **Curator** with read-write access who owns structure and currency, and a **Navigator** with read-only access who consumes it. Separating them keeps the corpus from being quietly reshaped by whoever last needed an answer. The honest trade-off: somebody has to build the routing tables. Vector RAG is automatic but dumber; structured routing is manual but dramatically more accurate for domain-specific Q\&A. If the corpus is large, general, and nobody will own it, this is the wrong tool. The rest of this page is how the routing is built. ## The Agent-Consumption Weighting Not all documentation types are equally useful to agents. Based on [Diataxis](https://diataxis.fr/) (Daniele Procida), we weight the four document types for agent consumption: | Type | Agent Value | Why | | --------------- | ----------- | ---------------------------------------------------------------------------------------------------------------- | | **Reference** | Highest | Structured, predictable, greppable. Consistent format means the agent can parse reliably | | **How-to** | High | Action-oriented recipes map directly to agent tasks. Step-by-step instructions translate into actions | | **Explanation** | Medium | Provides context for judgment calls. But costs tokens proportional to discursiveness. Best accessed on-demand | | **Tutorial** | Low | Agents don't build confidence, learn by repetition, or benefit from "we" language. Almost entirely wasted tokens | This inverts the typical human documentation priority. Humans want tutorials first; agents want reference first. ## Directory Layout A KB serving both human and agent consumers: ``` knowledge-store/ ├── index.md # Entry point: routing table ├── reference/ # Agent-primary │ ├── api-changes.md │ ├── configuration.md │ └── error-codes.md ├── howto/ # Agent-primary │ ├── migrate-security.md │ ├── handle-deprecation.md │ └── configure-logging.md ├── explanation/ # Agent-secondary (on-demand) │ ├── why-api-changed.md │ └── design-rationale.md └── tutorials/ # Human-only (agent ignores) └── getting-started.md ``` ## The Index Pattern The `index.md` at every directory level is the agent's entry point. It contains a **routing table** — not content, but pointers: ```markdown theme={null} # Spring Migration Knowledge | Topic | File | Read when... | |-------|------|-------------| | Import changes | reference/javax-to-jakarta.md | Task involves import migration | | Security config | howto/migrate-security.md | Project uses Spring Security | | JPA changes | reference/jpa-changes.md | Task involves data access | | Why APIs changed | explanation/api-rationale.md | Agent needs design context | ``` The "Read when..." column is critical. It tells the agent *under what conditions* to read the file. This is more useful than a document type label — it encodes priority and relevance. ### Routing precedence * "Always read first" — mandatory context * "Task involves X" — conditional on the current task * "Only when stuck" — fallback for debugging ## Progressive Disclosure The agent reads in layers: \~50 lines. The agent sees what domains exist and which are relevant to its task. \~30 lines. The agent sees specific topics and their routing conditions. Full content — but only for the 1-3 files that match the task. Not the whole KB. A well-structured KB turns a 50-file knowledge base into 2-3 file reads. The agent spends tokens on knowledge, not navigation. ## Two KB Types The lab uses two distinct KB architectures: ### Code-Agent KB (task-driven) For agents that execute coding tasks. Optimized for lookup and action. * Root `index.md` ≤100 lines * `VOCABULARY.md` — controlled vocabulary for consistent terminology * Domain directories with per-domain `index.md` * Cheatsheets and structured reference files * **Update cadence**: when frameworks or tools change * **Agent roles**: Curator (read-write maintenance) + Navigator (read-only consumption) ### Research-Partner KB (question-driven) For research synthesis and strategic context. Optimized for understanding and connections. * `CLAUDE.md` as session bridge (routing + context) * `synthesis/` hierarchy with theme index and per-theme docs * Immutable source conversations * **Update cadence**: after each research conversation * **Agent role**: session bridge (one agent, dual modes — synthesis intake + Q\&A) Don't mix them. The same domain can appear in both KB types with different purposes. A code-agent KB about Spring Security has migration recipes. A research-partner KB about Spring Security has strategic analysis of the migration's impact on the product roadmap. ## Design Rules 1. **Index files contain pointers, not content.** If you're putting explanation in the index, it belongs in a separate file. 2. **Reference format should be greppable.** Consistent headings, predictable structure, machine-parseable tables. The agent's first retrieval is typically `Grep` for a keyword, then `Read` of the matching file. 3. **One topic per file.** A file that covers both "how to migrate security" and "why the security API changed" should be split. The agent might need one without the other. 4. **Negative knowledge is explicit.** If something is out of scope, say so in the index. "This KB does NOT cover: deployment, monitoring, performance tuning." This prevents the agent from searching fruitlessly. 5. **KnowledgeRefs are relative paths.** In experiment datasets, `knowledgeRefs` point to files relative to `knowledgeBaseDir`. Typically 1-5 directory refs per item (usually 2-3). The agent reads the pointed-to index, then drills down. ## Evidence ### Code Coverage v1 Variant 3 (flat knowledge base) vs Variant 4 (structured skills) — identical content, different packaging. Variant 4 outperformed Variant 3 in efficiency metrics. The agent using structured skills showed 0% JAR\_INSPECT — it stopped needing to inspect dependencies because the knowledge was delivered proactively. ### SkillsBench [SkillsBench](https://arxiv.org/abs/2602.12670) confirmed that structure matters: [AgentSkillOS](https://arxiv.org/abs/2602.12670) found that hierarchically structured skills outperform flat files even with identical content. ### Partial Knowledge Paradox Some knowledge without structure *decreases* performance (Code Coverage v1, finding #4). An unstructured KB is worse than no KB — the agent wastes tokens navigating and gets confused by contradictory or irrelevant information. ## Further Reading For a narrative walkthrough of how these patterns were discovered and applied across 1,772 files and six federated KBs, see [Look Ma, No RAG!](https://blog.pollack.ai/look-ma-no-rag/) on the blog. ## Related How knowledge stays true after it's written — drift, rituals, and the trust principle The methodology this KB pattern belongs to, and the `/forge-kb` command that builds one JIT Retrieval end to end — five papers, twenty minutes, a corpus you can query # Knowledge Base Freshness Source: https://lab.pollack.ai/methodology/knowledge-base-freshness How knowledge stays true after it's written — cached routing judgments, two-channel freshness, and rituals that consume drift signals ## The Problem A knowledge base is written once but consumed indefinitely. Every routing table, index row, count, and date is a *claim about the world* that was true at write time. The world moves: code evolves past the design doc that describes it, files get added without index rows, a count in a header drifts from the count on disk. [Knowledge Base Design](/methodology/knowledge-base-design) answers "how does an agent find the right file?" This page answers the follow-up that only shows up months later: **why should the agent trust what it finds?** ## The Failure Mode: Cached Routing Judgment The dangerous rot is not a broken link — link checkers catch those. It is a **cached judgment**: an entry point that encodes a decision — "this is the authoritative design doc," "all files are indexed," "this copy matches the live one" — that was correct when written and is silently wrong now. Every reader reuses the judgment without re-deriving it. Three instances from one health pass over a federated KB system: * A federation catalog stayed perfectly fresh through status ingestion — while a per-project entry document it routed to sat unchanged for three months. Readers got a fresh pointer to rotten content. * The script that polices index drift had itself drifted from its versioned copy. The checker was unchecked. * An agent-based review reported "all files indexed — PASS." A deterministic count minutes later found seven missing entries. The common thread: each artifact was trusted because of what it was *named* — the catalog, the checker, the index — not because anything verified it. ## Two-Channel Freshness A federated knowledge system stays fresh through two channels, and they fail differently: | Channel | Direction | What it keeps fresh | How it fails | | --------------------------- | ---------------------------- | -------------------------------------------------- | ------------------------------------------------------------ | | **Status ingestion** (push) | Satellite projects → catalog | The union catalog: what exists, what changed, when | The entry documents it routes to rot beneath a fresh catalog | | **Catalog routing** (pull) | Reader → catalog → KB | Nothing — it only consumes | It trusts entries no ritual has checked | The push channel updates the *map*; nothing in it re-verifies the *territory*. The corollary is an observable rot gradient: knowledge federated through a design document rots fastest, because the status channel keeps the pointer fresh while no ritual touches the document itself. ## Deterministic Floor, Semantic Judgment Layer the checks the same way a [four-tier jury](/methodology/four-tier-jury) layers evaluation — deterministic first, LLM last: * **The deterministic floor** — scripts with exit codes. Link resolution, count reconciliation (claimed vs. on disk), copy drift (live vs. versioned `diff`), version-control tracking checks. Cheap, repeatable, and immune to plausible-sounding summaries. * **Semantic judgment above it** — an agent pass over content claims: does the summary still match the source? Is the concept glossary complete? Valuable for what scripts can't see — but it can report PASS on things it didn't actually verify. The rule: **semantic judgment never substitutes for a deterministic check that could exist.** When an agent reports a bulk PASS, spot-check it deterministically. The "all files indexed" failure above was exactly this substitution. ## Rituals Consume Drift Signals The governing principle: > Any operationally important "latest truth" channel needs both a source of truth **and a ritual that is required to consume its drift signal.** Logs, warnings, stale dates, and versioned backups only matter if something must read them. A `last-updated` date in a header is not a freshness mechanism — it is a freshness *signal*, and a signal nothing is required to read is noise. The fix is a ritual: a recurring re-index pass whose checklist includes consuming the signals — running the drift script, reconciling the counts, advancing the dates, and treating any non-clean result as work. When the ritual finds drift, the sequencing rule is: 1. **Fix the immediate inconsistency first.** 2. **Then add the smallest deterministic machinery that prevents recurrence.** Building the checker while the data is still wrong produces a checker calibrated against a broken baseline. ## The Trust Principle > No entry point is trusted because of its filename. It is trusted because the ritual checks it. Naming conventions — `index.md`, a root routing table, a federation catalog — create *expectations* of authority, and expectations rot silently. Actual authority comes from being inside some check's blast radius. If a file is operationally important and no deterministic check or ritual step would notice it going stale, its trustworthiness is an accident of how recently someone happened to look. ## Design Rules 1. **Never duplicate state that lives in checked files.** Architecture and overview docs state invariants and flows, then *point* at the files where counts, dates, and lists live. A duplicated count is a second copy waiting to rot. 2. **Every claim a reader might act on is either generated or checked.** If it's neither, delete it or move it somewhere advisory. 3. **Checkers are channels too.** Drift detectors have copies; re-index procedures have versions. Include the checking machinery itself in the check surface — the unchecked checker is the failure mode that hides longest. 4. **Prefer reconciliation over re-assertion.** A check that compares two independent sources (index vs. disk, claimed count vs. computed count, live copy vs. versioned copy) finds drift. A check that re-reads one source merely re-caches its judgment. ## Connection to the Flywheel This is the [improvement flywheel](/methodology/improvement-flywheel) applied to the knowledge layer itself. The KB is an agent artifact like any other: it has measurable gaps (drift signals), diagnostic lenses (deterministic checks and semantic passes), and targeted interventions (fix, then smallest machinery). A knowledge base that nothing measures degrades exactly the way an agent that nothing judges does — invisibly, and with full confidence. ## Related Structure for finding the right file — the write-time half of the problem The same deterministic-first layering, applied to evaluating agent output Measured gaps → targeted interventions — here applied to knowledge infrastructure # Knowledge-Directed Execution Source: https://lab.pollack.ai/methodology/knowledge-directed-execution The thesis: knowledge + structured execution > model ## The Thesis > **Knowledge + structured execution > model** Agent reliability improves more from giving agents the right knowledge and constraining their execution than from switching to a larger model. ## What This Means ### Knowledge Not "more data" — **curated, structured domain knowledge** delivered to the agent at the right time: * Which testing patterns work for this framework * What dependencies are available and how to use them * What the project conventions are * What common failure modes look like ### Structured Execution Not "better prompts" — **infrastructure that shapes agent behavior**: * Deterministic preprocessing before the LLM acts * Tool configuration that guides tool selection * Execution loops with built-in checkpoints * Judge feedback that catches failures early ### > Model This doesn't mean models don't matter. It means that for a given model, you get more reliability improvement from knowledge and execution infrastructure than from upgrading to the next model tier. ## Agents Are Workflows Before the thesis can be tested, the object it applies to has to be named honestly. An agent is not a magic black box — it is a **workflow**. Each step is either **deterministic** (build, lint, test, measure coverage) or an **AI step** (reason about an error, generate code, decide what to fix next). What people casually call "an agent" is usually just the AI step — one node in a larger pipeline. ``` fetch PR → rebase → detect conflicts → run tests → fix & retest → cleanup → build gate [determ.] [determ.] [deterministic] [determ.] [AI step] [determ.] [judge] │ ┌────────────────────────┐ │ pass │ version-pattern judge │◄───┘ │ [deterministic] │ └──────────┬─────────────┘ │ ┌──────────┴─────────────┐ │ parallel AI steps │ │ assess-code-quality │ │ assess-backport │ └──────────┬─────────────┘ │ ┌──────────┴─────────────┐ │ quality judge → report│ └────────────────────────┘ ``` This is the actual [AgentWorks PR Review](https://github.com/markpollack/agentworks-pr-review) pipeline — seven deterministic steps, a judge gate, then parallel AI assessment only if the build passes. Most of the workflow never touches an LLM. That is what makes the thesis actionable rather than rhetorical. If an agent were a single model call, "knowledge + structured execution > model" would have nowhere to apply. Because it is a workflow, every deterministic step is a source of variance removed, and every AI step is a place where curated knowledge changes the outcome. ## Evidence ### Code Coverage v1 The [first experiment](/experiments/code-coverage-v1) showed two independent axes of improvement — knowledge injection and prompt hardening — both of which operate on infrastructure, not model choice. The PetClinic "model floor" (92-94% coverage regardless of variant) demonstrates that the model's prior knowledge creates a ceiling that only infrastructure can differentiate. ### SkillsBench (External) [SkillsBench](https://arxiv.org/abs/2602.12670) (Feb 2026) found that 2-3 curated skills improve agent performance by +16.2 percentage points on average. Comprehensive skills actually *decrease* performance by -2.9pp. This validates "curated > comprehensive" — structure matters. ### Stripe Convergence Stripe arrived at the same structure independently, at a scale that makes it hard to dismiss. Their **Minions** system merges over a thousand pull requests a week containing no human-written code, against hundreds of millions of lines of Ruby on a platform handling more than \$1 trillion in annual payment volume. They call the pattern **blueprints** — deterministic steps interleaved with agent reasoning steps, which is the diagram above under a different name. As Stripe's Alistair Gray puts it: *"Blueprints combine the determinism of workflows with agents' flexibility in dealing with the unknown."* The detail that matters most for this thesis is *why* they needed it: their stack is uncommon and their libraries are homegrown, so the model does not know their codebase. Rules files supply the knowledge and blueprints supply the structure — knowledge and execution structure compensating for what the model does not have. Two independent groups converging on interleaved determinism is better evidence than either alone. Sources: [Minions: Stripe's one-shot, end-to-end coding agents](https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents) (Alistair Gray, February 2026), and Anup Jadhav's [The walls matter more than the model](https://www.anup.io/stripes-coding-agents-the-walls-matter-more-than-the-model/). ## The Equation ``` Agent Reliability = f(Knowledge Quality × Execution Structure × Model Capability) ``` Current industry focus is almost entirely on Model Capability. This lab focuses on the first two terms, where the marginal returns are higher. ## Naming History This concept has gone through several names: | Name | Status | | ----------------------------------------- | ------------------------------------------------------ | | "Infrastructure over prompts" | Early framing, too narrow | | "Knowledge-directed execution" | Current, captures both components | | "Curated opinions + structured execution" | Verbose but precise | | "The walls matter more than the model" | Anup Jadhav's phrasing for the Stripe result, resonant | ## How to Apply If you're building agent systems: 1. **Start with knowledge** — What does your agent need to know that it doesn't? 2. **Structure the delivery** — Skills > flat files > nothing 3. **Add execution constraints** — Deterministic preprocessing, judge feedback loops 4. **Then consider the model** — Upgrade only after infrastructure is solid # Agento University Source: https://lab.pollack.ai/product/agento-university A visual operating environment for a fleet of agents that each own a codebase
IN DEVELOPMENT Exploratory — product vision
A team of Agenti — thinker, field agent, professor, graduate, and greeter
Give every repository an agent that owns it. Then give yourself somewhere to watch them all.
Agento University is the graphical surface for the steward pattern — a workspace for the agent that owns one codebase, and a campus view for the fleet that owns the rest.
## The Problem It Solves The [Forge methodology](/methodology/forge) is a command-line product. You run a slash command, an agent scaffolds a vision or works a roadmap step, and the result is files in git. That works, and it is how the methodology is used today. It does not scale to *watching*. Once a dozen repositories each have an agent continuously accountable for them, the questions stop being "what did this session do" and start being: which of these needs me right now? What is that one waiting on? What did it change while I was gone? A terminal answers those one repository at a time. Agento University is the answer to the second question. It is not a different methodology — it is a **higher-level interface onto the same stewards**, aimed at the point where a fleet becomes too large to hold in your head. ## Two Surfaces ### The Studio — working with one agent The Studio is the workspace for a single steward. It puts four things in one shell: * **The workflow canvas** — a rendered view of the agent's workflows. Clicking a node opens the exact line of source that declared it, so the picture and the code never drift apart. * **The artifact panel** — the agent's documents and files, with a viewer that opens at a line and highlights it. * **The inbox** — the agent's message lanes, so obligations arriving from elsewhere have a human end. * **The action bar** — run a turn; or forge a new steward into existence and start its first conversation, without leaving the shell. ### The Campus — watching the fleet The campus is the fleet view: each repository is a building, each building carries a live status billboard, and the state you are looking at is written by the stewards themselves rather than assembled by a separate monitoring service. The metaphor is a navigation aid, not a theory. A building is a repository; a billboard is that repository's current state. Where the metaphor stops being useful, the interface uses ordinary words.
SCREENSHOTS PENDING
Captures of the Studio shell, the workflow canvas, the inbox, and the campus billboards are being prepared from the running application.
## What Is Actually Built The shell is real and tested; the product thesis on top of it is not yet proved. Stating that boundary is more useful than a launch date. | Surface | State | | ---------------------------------------------------------------------- | --------------------- | | Studio shell — sidebar, advisor tabs, artifact panel | **Built** | | Workflow canvas with click-through to declaring source | **Built** | | Inbox — message lanes with a human end | **Built** (read-only) | | Action bar — run a turn, forge a new steward | **Built** | | Campus — buildings, billboards, interiors | **Built** | | Review-loop cockpit — the product thesis | In progress | | Runtime surfaces — live run watching, approvals, experiment dashboards | Deferred | ## On "Graduation" The campus metaphor supplies a word that turns out to carry a real engineering meaning, so it is worth stating plainly rather than leaving as a pun. **An agent graduates when it reliably meets a repeatable benchmark.** Not when it produces one good result — when the judges that scored its development converge, stop being development scaffolding, and become a measuring stick it clears consistently. That is a threshold you can compute, and it is the same threshold the [four-tier jury](/methodology/four-tier-jury) exists to measure. Everything else about the campus is navigation. This part is not. ## Where It Sits Agento University is **exploratory and long-term**. It is deliberately not part of the control-loop architecture: the [Improvement Flywheel](/methodology/improvement-flywheel) measures and steers a single agent, and it does that whether or not any of this exists. The University is a layer above — coordination and visibility across many agents, once the single-agent problem is solved well enough to have many. The projects it is built on — [Agent Workflow](/projects/agent-workflow) as the engine, [Agent Journal](/projects/agent-journal) for traces, [Agent Judge](/projects/agent-judge) for evaluation — are shipped and documented today. Agento University is the product surface being assembled on top of them. ## Related What a steward is, and what the University is a fleet view of The methodology and the command-line product underneath # ACP Java SDK Source: https://lab.pollack.ai/projects/acp-java-sdk Agent Communication Protocol — build agents, consume agents, integrate with IDEs using a standard protocol **[What's New →](/docs/acp-java-sdk/whats-new)** Pure Java implementation of the Agent Communication Protocol (ACP) — a standard for agent-to-agent and agent-to-IDE communication. Build agents that any ACP-compatible IDE can connect to, or consume agents from any ACP-compatible runtime. Three-phase lifecycle: Initialize → Session → Prompt. Three agent API styles: sync, async, and annotation-driven. ACP is the planned wire protocol for `A2AStep` in [Agent Workflow](/projects/agent-workflow) — wrapping a remote agent as a local `Step`. **Current release: 0.16.1.** **0.15.0:** notifications now arrive in order and survive a graceful close; Jackson 2.21.5 and Jetty 12.0.37 clear 17 known advisories in the published closure; the verbatim Apache-2.0 license ships inside every artifact. No protocol or public API changes — a drop-in upgrade from 0.14.0. ## Getting Started Client SDK, agent SDK, test utilities — three-phase lifecycle, three API styles Launch an agent subprocess, create a session, send your first prompt ## Tutorials Progressive modules across multiple parts: Protocol basics, sessions, prompts, streaming updates, agent requests, permissions, error handling Echo agent (25 lines), handlers, sending updates, agent requests, in-memory testing, capability negotiation, MCP servers Async client with Project Reactor, async agent, reactive patterns Zed, JetBrains, and VS Code integration guides ## Reference Client classes, agent classes, protocol types, transports, error codes SDK implementation — client and server support Runnable tutorial modules with integration tests ## Role in the Lab * **[Agent Workflow](/projects/agent-workflow)** — planned `A2AStep` will wrap remote ACP agents as local `Step` in workflows * ACP standardizes how agents discover, connect, and delegate — the communication backbone for multi-agent architectures * IDE integration enables agents built with Agent Workflow to run inside Zed, JetBrains, and VS Code # Agent Bench Source: https://lab.pollack.ai/projects/agent-bench Benchmarking suite for Java-centric AI agents on real-world software engineering tasks **[What's New →](/docs/agent-bench/whats-new)** The latest public release is 0.6.1 under the `io.github.markpollack` Maven group and Java package namespace. ## Overview Agent Bench measures AI coding agents on repeatable software-engineering tasks. Inspired by Terminal-Bench, each benchmark separates the task definition, the agent that changes a workspace, and the verifier that grades the result. Agent Bench uses [Agent Judge](/projects/agent-judge) to materialize deterministic checks and cascaded juries rather than reimplementing grading. The filesystem is the contract: any CLI agent that reads `INSTRUCTION.md` and modifies the workspace can participate without an SDK integration. ## Maven Artifacts Use the core module for the agent-neutral CLI and Java API: ```xml theme={null} io.github.markpollack agent-bench-core 0.6.1 ``` Use the agents module when `BenchApp` and the real Agent Client-backed LLM judge are required: ```xml theme={null} io.github.markpollack agent-bench-agents 0.6.1 ``` ## Supported Usage Patterns `run` prepares a workspace, executes setup and the configured agent, runs post-processing, and grades the result. `provide` prepares the files, an external agent operates on them, and `grade` evaluates the resulting workspace. Use the benchmark catalog, command classes, `JudgeFactory`, Agent Judge types, and result model directly from Java. ## Execution and Trust Boundary Benchmark setup scripts, post-processing scripts, and configured agent commands execute as local host processes with the permissions of the user who starts Agent Bench. A workspace directory organizes inputs and results, but it is not a security boundary. Agent Bench does not provide container execution or provision isolation. Run untrusted benchmark definitions or agents only inside your own disposable VM, CI runner, or other externally managed isolation. ## CLI Entry Points * `agent-bench-core` runs `BenchMain`, the agent-neutral CLI with deterministic Agent Judge checks. * `agent-bench-agents` runs `BenchApp`, which retains the core commands and wires the real Agent Client-backed LLM judge for LLM-graded `run` and `resume` operations. ## Quick Start from Source ```bash theme={null} git clone https://github.com/markpollack/agent-bench.git cd agent-bench git checkout v0.6.1 ./mvnw clean verify ./mvnw -q -pl agent-bench-core exec:java \ -Dexec.args="run --benchmark hello-world --agent agents/my-agent.yaml" ``` ## Benchmarks | Benchmark | Status | What it measures | | ----------------------- | ------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | **hello-world** | Working | File creation and basic agent infrastructure | | **code-coverage** | Working judges; full agent run requires provider access | JUnit test generation, coverage uplift, and LLM-scored practice adherence | | **spring-boot-upgrade** | Candidate | Spring modernization; the public jury is a build quick-check and the baseline-aware grader is owner-operated externally | ## Documentation Run end-to-end, split, and Java API workflows Configure a CLI tool and understand its host permissions Agent Judge tiers, built-in judges, and custom judge types Core and agents-module commands ## Source Released 0.6.1 source --- two published modules # Agent Client Source: https://lab.pollack.ai/projects/agent-client Portable Java API for autonomous CLI agents — actively verified with Claude Code, Codex, and Gemini CLI **[What's New →](/docs/agent-client/whats-new)** This project has moved from `spring-ai-community/agent-client` to `markpollack/agent-client`. The Maven groupId changed from `org.springaicommunity.agents` to `io.github.markpollack`. Portable Java API for autonomous CLI agents. No Spring Boot required — build a model, create a client, run a goal. Optional Spring Boot starters for auto-configuration. Agent Client is the bridge between raw SDK calls ([Claude Agent SDK](/projects/claude-agent-sdk)) and higher-level orchestration ([Agent Workflow](/projects/agent-workflow)). **Version 0.29.3** — [Maven Central](https://central.sonatype.com/search?q=g:io.github.markpollack+a:agent-client-core) **0.29.0:** Provider trajectories through the facade. Grok, Codex, and Antigravity now publish their parsed run trajectory through `AgentClientResponse.getPhaseCapture()`, matching the Claude facade contract — Grok from its native `streaming-json` ACP stream, Antigravity from `stream-json`, and Codex by harvesting its durable rollout JSONL after execution. Live facade gates verify all three return a capture containing tool uses. The adapters consume the released Agent Journal 1.8.0 capture modules, so `agent-model` and the production surface of `agent-client-core` take on no journal dependency. LOOSE-mode Codex execution is repaired: full-auto now maps to the global `--sandbox workspace-write` and `--ask-for-approval never` options and no longer implicitly grants full-disk access. All 29 modules are published, `agent-tck` included. **0.28.0:** Two new CLI providers. `agent-grok` / `grok-cli-sdk` / `agent-starter-grok` for Grok, whose headless JSON envelope yields a read rather than scraped session id, token usage, and per-run USD cost, and whose caller-supplied session UUID makes resume possible without recovering an id from output. `agent-antigravity` / `antigravity-cli-sdk` / `agent-starter-antigravity` for Google's `agy`, which declares its working directory with `--add-dir` so the CLI cannot divert writes to a shared scratch directory while reporting success, and derives run success from whether work was produced and nothing refused rather than from a `status` field that reports ERROR on complete and correct responses. Both join the parity TCK and pass all ten scenarios against live CLIs. **0.27.0:** Published-consumer correction. Direct Jackson and Log4j declarations now survive flattened child POMs, a generated no-parent/no-BOM gate verifies all 23 public runtime modules, and the compatible Claude SDK 1.5.0, Agent Journal/Capture 1.7.0, and Agent Sandbox Core 0.10.0 train is adopted. Current source/archive licensing is aligned with the Mark Pollack BSL distribution and retained Apache history. **0.26.0:** Breaking diligence cleanup. Removes the abandoned Vendir context advisor and Git-repository DSL, retires the stale `agents-runtime` container build and Docker TCK, and pins release automation. Claude Code, Codex, and Gemini CLI are the actively verified provider set; other adapters remain experimental. Its immutable Maven Central child POMs do not preserve all reviewed dependency floors; use 0.27.0 for the published-consumer correction. **0.25.0:** Fixes Spring Boot auto-configuration for all providers — the org migration had left the auto-config registration files pointing at the old `org.springaicommunity` package, so the starters failed to auto-configure (`ClassNotFoundException`) at boot. No public API changes. **0.24.0:** Compatibility fix — `agent-claude` now builds against agent-journal 1.6.0, restoring Claude trace-wiring for Spring Boot consumers on agentworks-bom 1.12.0+ (journal 1.5.0 had relocated `TraceContentMode`). No public API changes. **0.23.0 highlights:** Upgraded to Spring AI 2.0.0 GA on Spring Boot 4.0.7, clearing transitive Boot and Spring AI CVEs. Spring AI 2.0 moves to Jackson 3, so Jackson 2 usage is now declared explicitly. Default Gemini model is now `gemini-3.5-flash`. Still current from 0.21.0: portable reasoning effort (`low`/`medium`/`high`) via `AgentOptions.getEffort()`, with provider-native overrides for Claude (`--effort`, up to `max`) and Codex (`model_reasoning_effort`, up to `xhigh`). Dependency: `claude-code-sdk` 1.4.0. ## Architecture Three layers, each usable independently: | Layer | Module | Framework Deps | | --------------------------- | ---------------------------------------- | ----------------- | | **Core API** | `agent-client-core` | None — plain Java | | **Spring Boot Auto-Config** | `agent-client-spring-boot-autoconfigure` | Spring Boot | | **Starters** | `agent-starter-claude`, etc. | Spring Boot | ```java theme={null} // Plain Java — no Spring Boot needed ClaudeAgentModel model = ClaudeAgentModel.builder() .defaultOptions(ClaudeAgentOptions.builder().yolo(true).build()) .build(); AgentClient client = AgentClient.create(model); AgentClientResponse response = client.run("Create hello.txt"); ``` ## Providers Claude Code, Codex, Gemini CLI, Grok, and Antigravity all pass the provider parity TCK — ten scenarios each, zero skips, against live CLIs. CI re-verifies Claude Code, Codex, and Gemini on every commit; Grok and Antigravity are verified against live CLIs but cannot run in CI, because both authenticate interactively and cache credentials rather than reading an API key. Amazon Q, Amp, Qwen Code, and SWE-agent carry no parity coverage and remain experimental. Provider selection happens at construction time — everything after `AgentClient.create(model)` uses the shared client contract. ## LOOSE / STRICT Modes `AgentClientMode` controls default permissiveness. LOOSE (default) bypasses sandbox restrictions and git checks for frictionless evaluation. STRICT requires explicit opt-in. ## Documentation Plain Java quick start — create your first agent task Step-by-step lessons from first task to multi-provider 18 configuration options, trace files, authentication Configuration precedence, LOOSE/STRICT modes ## Source Source code, examples, and getting started guide ## Related Isolated execution — local, Docker, or E2B cloud Uses Agent Client for `ClaudeStep` and `AgentClientStep` # Agent Experiment Source: https://lab.pollack.ai/projects/agent-experiment Reproducible Java experiment lifecycle for evaluating AI agents **[What's New →](/docs/agent-experiment/whats-new)** **Current release: 0.7.1** · Java 21 · three modules ## Overview Agent Experiment provides a repeatable lifecycle for agent evaluations: load versioned fixture or Git-backed datasets, provision isolated workspaces, invoke an agent, judge results with [Agent Judge](/projects/agent-judge) juries, persist structured evidence, and compare variants. `experiment-core` has no direct agent-SDK integration. `experiment-claude` adds Claude Code SDK invocation, planning, and semantic evaluation, while `experiment-workflow` adapts typed Agent Workflow executions and journaled step costs to the common `AgentInvoker` contract. The framework records source and dataset revisions, dirty state, configuration, knowledge hashes, workspaces, results, costs/tokens, and journals. Exact replay still depends on callers pinning the external model, CLI, tools, network services, and other nondeterministic inputs. ## Architecture Git-managed fixture datasets with items, before/reference snapshots, and version tracking Orchestrates the full loop: load items, invoke agent, judge, aggregate, persist Compare runs across variants with per-judge deltas, regression detection, and summary statistics Group variant results into sessions, group sessions into sweeps for multi-run analysis Post-hoc re-scoring of stored results without re-invoking agents Run a judge as the system under test against labeled datasets ## Modules | Module | Description | Key Dependencies | | --------------------- | -------------------------------------------------------------------------------- | ------------------------------------------------ | | `experiment-core` | Datasets, workspace provisioning, runner, journals, comparison, results, storage | Agent Judge 0.15.1, Agent Journal 1.8.2, Jackson | | `experiment-claude` | Claude SDK invoker, plan generator, semantic judge | Claude Code SDK 1.5.1 | | `experiment-workflow` | Typed workflow invokers with journal and cost adaptation | Agent Workflow 0.12.1, Agent Client 0.29.3 | ## Documentation Run your first experiment: dataset, agent, jury, variant comparison Design datasets, configure variants, wire custom judges Build cascaded juries for tiered evaluation Core types, runner, comparison, storage, diagnostics ## Quick Start ```xml theme={null} io.github.markpollack experiment-core 0.7.1 ``` ```java theme={null} ExperimentConfig config = ExperimentConfig.builder() .experimentName("my-experiment") .datasetDir(Path.of("datasets/my-benchmark")) .model("sonnet") .promptTemplate("Your task: {{task}}") .perItemTimeout(Duration.ofMinutes(10)) .build(); DatasetManager dm = new FileSystemDatasetManager(); ResultStore store = new FileSystemResultStore(resultsDir); AgentExperiment experiment = new AgentExperiment(dm, jury, store, config); ExperimentResult result = experiment.run(invoker); // result.passRate(), result.totalCostUsd(), result.items() ``` The [public template](https://github.com/markpollack/agent-experiment-template) provides a complete starting point with variant configuration, workflow invokers, default-on journal capture, and a credential-free smoke test. ## Release and Compatibility Version 0.6.0 moves runtime judging to Agent Judge 0.14.0. Stored results use Agent Experiment-owned `RecordedJudgment` and `RecordedVerdict` projections. Existing 0.5 / Agent Judge 0.13 result files load automatically and preserve normalized outcomes, reasoning, checks, labels, metadata, and available composite evidence. Re-saving writes the new format; obsolete range bounds, categorical allow-lists, and unnamed legacy composite identity cannot be reconstructed losslessly. Standalone consumers resolve Agent Journal/Capture 1.8.2, Jackson 2.22.2, and Jackson 3.2.2 without an AgentWorks BOM. The parent artifact publishes one aggregate CycloneDX 1.6 JSON SBOM, and the stable release includes signed binaries, sources, and Javadocs with the project BSL text. ## Role in the Lab Agent Experiment is the execution layer that ties the other AgentWorks projects together: * **[Agent Judge](/projects/agent-judge)** — Jury scores every item * **[Agent Journal](/projects/agent-journal)** — Traces captured during invocation * **[Agent Sandbox](/projects/agent-sandbox)** — Isolated execution environments * **[Agent Bench](/projects/agent-bench)** — Benchmark datasets consumed by experiments Public experiments built on this lifecycle include: * [Code Coverage v1](/experiments/code-coverage-v1) * [Code Coverage v2](/experiments/code-coverage-v2) ## Source Source code (BSL 1.1) — three modules, 544 tests # Agent Hooks Source: https://lab.pollack.ai/projects/agent-hooks Portable hook API for steering agent behavior at the tool-call boundary — write once, run on any runtime **[What's New →](/docs/agent-hooks/whats-new)** **Current release: 0.8.2.** **0.7.0:** Built on Spring AI 2.0.0, Spring Boot 4.0.7, and Claude Agent SDK 1.5.0. This release keeps the core-plus-adapters architecture and public dispatch API while correcting standalone dependency resolution, packaging, diagnostics, and Java compatibility claims. Every agent framework implements hooks differently. Claude Code has shell-based hooks. Strands has steering callbacks. Spring AI has advisors. Your safety policy, logging, and steering logic gets rewritten for each one. Agent Hooks is a portable Java API that lets you write hook logic once and run it on any runtime that has an adapter. The core module has no framework dependency — it defines the event model, decision types, and registry, with `org.jspecify` annotations as its only compile dependency. Adapters (Spring AI, Claude Agent SDK, and Gemini CLI) wire the core into their runtime's tool-call lifecycle. Your hooks move with you when your agent infrastructure changes. ## Why Hooks LLMs are probabilistic — prompt-based instructions drift under token pressure. Agents skip steps, forget constraints, and ignore guardrails. Hooks solve this by moving critical logic out of the prompt and into deterministic code that intercepts every tool call, the same way servlet filters intercept HTTP requests: * **Safety** — Block dangerous operations before they execute. A `Block` decision short-circuits immediately and cannot be overridden by later hooks. * **Observability** — Log every tool call, capture timing data, and feed traces into [Agent Journal](/projects/agent-journal) for behavioral analysis. * **Steering** — Modify tool inputs in flight. Subsequent hooks see the modified input, so transformations chain cleanly. ## How It Works Hooks intercept at two points in the tool-call lifecycle: ``` BeforeToolCall ──► Tool Executes ──► AfterToolCall │ │ Block? ◄── short-circuit Retry? Modify? ◄── chains Log / observe Proceed ◄── default Cleanup (reverse order) ``` Register hooks with type-safe generics and optional tool-name filtering: ```java theme={null} // Block all shell commands registry.onTool("shell.*", BeforeToolCall.class, event -> HookDecision.block("Shell access disabled in this environment")); // Log every tool call registry.on(AfterToolCall.class, event -> { log.info("{} completed in {}ms", event.toolName(), event.duration().toMillis()); return HookDecision.proceed(); }); // Redirect file writes to a sandbox directory registry.onTool("write.*", BeforeToolCall.class, event -> { String sandboxed = event.toolInput() .replace("/home/user", "/sandbox"); return HookDecision.modify(sandboxed); }); ``` ## Decision Model `HookDecision` is a sealed type with four variants: | Decision | When | Behavior | | --------- | -------------------- | ----------------------------------------------------------------------------- | | `Proceed` | Default | Tool executes normally | | `Block` | Safety / policy | Short-circuits immediately — later hooks never run | | `Modify` | Input transformation | Passes modified input to the next hook in the chain | | `Retry` | AfterToolCall only | **Advisory.** Returned to the caller; no shipped adapter re-executes the tool | Multiple hooks execute in priority order (default: 100, lower = earlier). AfterToolCall hooks fire in reverse priority order for proper cleanup semantics. Two behaviors are worth knowing before you rely on them. Dispatch matches an event's **exact runtime class**, so a hook registered against a supertype such as `ToolEvent.class` compiles but never fires — register against the concrete record. And a hook that *throws* is logged at `WARNING` and treated as `Proceed`: `Block` short-circuits, but a failing hook does not fail closed. ## Modules | Module | What it does | Dependencies | Java | | -------------------- | ---------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ------ | | `agent-hooks-core` | Portable API — events, decisions, registry | No framework dependency; `org.jspecify` annotations only | 17 | | `agent-hooks-spring` | Spring AI adapter — wraps `ToolCallback` with hook dispatch, auto-configures via Boot | Spring AI, Spring Boot | 17 | | `agent-hooks-gemini` | Gemini CLI adapter — stateless stdin/stdout dispatcher for Gemini's subprocess-per-event model | Jackson (compile) | 17 | | `agent-hooks-claude` | Claude Agent SDK adapter — bridges hook providers to Claude CLI hooks via `AgentHookBridge` | Claude Agent SDK (provided) | **21** | `agent-hooks-claude` requires Java 21 because every published `claude-code-sdk` version is Java 21 bytecode. The other three modules are unaffected on Java 17. ## Event Hierarchy The event system is open (unsealed) — you can define custom events for your runtime: | Event | Interface | Decisions | | --------------------------- | ----------- | --------------------------------- | | `BeforeToolCall` | `ToolEvent` | Proceed, Block, Modify | | `AfterToolCall` | `ToolEvent` | Proceed, Retry | | `SessionStart` | `HookEvent` | Observation only | | `SessionEnd` | `HookEvent` | Observation only | | `UserPromptSubmit` | `HookEvent` | Observation only (Claude adapter) | | `AgentStop` | `HookEvent` | Observation only (Claude adapter) | | `SubagentStop` | `HookEvent` | Observation only (Claude adapter) | | `PreCompact` | `HookEvent` | Observation only (Claude adapter) | | `GeminiBeforeAgent` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiAfterAgent` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiBeforeModel` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiAfterModel` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiBeforeToolSelection` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiNotification` | `HookEvent` | Observation only (Gemini adapter) | | `GeminiPreCompress` | `HookEvent` | Observation only (Gemini adapter) | ## Quick Start ```xml theme={null} io.github.markpollack agent-hooks-core 0.8.2 io.github.markpollack agent-hooks-spring 0.8.2 ``` ```xml theme={null} io.github.markpollack agent-hooks-claude 0.8.2 ``` ```xml theme={null} io.github.markpollack agent-hooks-gemini 0.8.2 ``` ## Write Once, Run Anywhere The same `AgentHookProvider` works on all three runtimes — this is the core value proposition. ```java theme={null} // This provider works on Spring AI, Claude CLI, and Gemini CLI public class SecurityHooks implements AgentHookProvider { @Override public void registerHooks(AgentHookRegistry registry) { registry.onTool("Bash", BeforeToolCall.class, event -> HookDecision.block("Shell access not permitted")); } } ``` **Spring AI** — register as a `@Component` bean, auto-configuration handles the rest: ```java theme={null} @Component public class MySecurityHooks extends SecurityHooks {} ``` The auto-configured `HookContext` is an **application-wide singleton**: every hooked tool call in the application shares one state map and one tool-call history. That suits a single-user CLI or desktop app. A multi-user server should define its own request- or session-scoped `HookContext` bean, or build one per conversation and call `HookedTools.wrap` at that point. **Claude Agent SDK** — bridge into the Claude `HookRegistry`: ```java theme={null} AgentHookRegistry registry = new AgentHookRegistry(); registry.register(new SecurityHooks()); AgentHookBridge bridge = new AgentHookBridge(registry); bridge.registerInto(claudeHookRegistry); ``` The bridge registers callbacks for all six Claude hook events (PreToolUse, PostToolUse, UserPromptSubmit, Stop, SubagentStop, PreCompact). It converts Claude SDK types to core events, dispatches through your hooks, and maps decisions back to Claude's `HookOutput`. Tool call duration is tracked via wall-clock timing across the pre/post hook boundary. Each Claude session gets its own `HookContext` for isolated state and history. The bridge retains those contexts for its own lifetime — fine for one bridge per CLI process; a long-lived bridge shared across sessions should call `evictSession(sessionId)` when a session finishes. **Gemini CLI** — stateless subprocess dispatcher reads JSON from stdin: ```java theme={null} public class MyGeminiHooks { public static void main(String[] args) throws Exception { GeminiHookDispatcher.create(new SecurityHooks()) .run(); // reads stdin, dispatches, writes stdout, exits } } ``` Gemini CLI spawns the hook process per event. The dispatcher maps all 11 Gemini events to core and Gemini-specific `HookEvent` records. `HookContext` is fresh per invocation — stateless hooks (security gates, audit logging) work out of the box. Note: Gemini BeforeTool can only allow or block — `Modify` is downgraded to allow with a warning. ## Documentation Core API, Spring AI adapter, Claude adapter, and Gemini adapter Architecture decisions, event hierarchy, dispatch semantics ## Suite integration Agent Hooks is managed by the [AgentWorks BOM](/projects/agentworks-bom), but it is standalone today: no other published AgentWorks module depends on it. Bridges to [Agent Journal](/projects/agent-journal) (a hook provider that logs tool-call events to a journal Run) and to workflow steps are intended directions, not shipped code. # Agent Journal Source: https://lab.pollack.ai/projects/agent-journal Structured execution events, portable agent traces, and derived analysis for Java agent research **Current release: 1.8.2** · **[What's New →](/docs/agent-journal/whats-new)** Agent Journal is a Java library for recording agent runs as typed events and analyzing them after execution. It stores immutable execution facts separately from derived analysis, and it can project Claude Code, Gemini CLI, Grok, Codex, and Antigravity results into a shared journal model. Release 1.8.0 adds tool-trajectory capture for Grok, Codex, and Antigravity, each parsed from that CLI's own durable JSONL stream into ordered tool and cost events. The three new modules carry no vendor SDK dependency and run on Java 17. The change is additive on the existing event and trace contracts. LLM calls, tool calls, state changes, metrics, git operations, and custom events in append-only JSONL. Cost attribution and step outcomes in a separate `analysis.jsonl` stream. `EvalSubject` adapters expose recorded behavior to evaluators such as Agent Judge. ## Modules | Module | Purpose | Runtime requirement | | ------------------------- | ---------------------------------------------------------------------------------- | ------------------- | | `journal-core` | Runs, events, storage, metrics, feedback, evaluation subjects, and portable traces | Java 17+ | | `claude-code-capture` | Claude Code SDK parsing and recording | Java 21+ | | `gemini-cli-capture` | Gemini CLI result parsing and recording | Java 21+ | | `grok-cli-capture` | Grok CLI trajectory parsing and recording | Java 17+ | | `codex-cli-capture` | Codex CLI trajectory parsing and recording | Java 17+ | | `antigravity-cli-capture` | Antigravity CLI trajectory parsing and recording | Java 17+ | `journal-core` uses Jackson 2 and the SLF4J API. The capture modules also depend on their respective Java SDKs, whose published bytecode requires Java 21. ## Install ```xml theme={null} io.github.markpollack journal-core 1.8.2 ``` The six released artifacts use the `io.github.markpollack` group ID and are available from [Maven Central](https://central.sonatype.com/search?q=g%3Aio.github.markpollack%20AND%20a%3Ajournal-core). ## Documentation Record and persist a first run with `journal-core`. Understand experiments, runs, event streams, analysis streams, feedback, and storage. Project Claude Code and Gemini CLI results into Agent Journal. Query generated journals with DuckDB. Stable 1.8.2 entry points, events, storage operations, and capture options. Journal files and trace files can contain prompts, tool inputs and results, file contents, and command output. Treat them as potentially sensitive operational data. `JsonFileStorage` is local file storage, not a multi-tenant security boundary. ## Related projects * [Agent Workflow](/projects/agent-workflow) provides the separate `workflow-journal` integration module. * [Agent Judge](/projects/agent-judge) consumes evaluation subjects derived from recorded behavior. * Markov fingerprinting is one downstream analysis technique for tool-call sequences. Agent Journal on GitHub (BSL 1.1) # Agent Judge Source: https://lab.pollack.ai/projects/agent-judge Framework-neutral evaluation for AI agent output across Spring AI, LangChain4j, Koog, and CLI agents **[What's New in 0.15 →](/docs/agent-judge/whats-new)** Agent Judge is a Java 21 evaluation library for deciding whether an agent result satisfies its goal. It turns a definition of done into executable evidence: builds and commands, coverage and class versions, semantic file checks, RAG evaluation, bounded LLM assessment, and independent judges combined through explicit jury policies. It asks whether the work meets its requirements—not whether it resembles one reference answer. The central result is status-first. Every `Judgment` has a `PASS`, `FAIL`, `ABSTAIN`, or `ERROR` status, while a normalized score and classification label are independent optional facts. Result metadata is restricted to portable values so judgments can cross JSON and process boundaries reliably. ## What you can verify Run commands and builds, check bytecode versions, and preserve or improve test coverage. Check files and compare Java, Maven, XML, and text by structure or meaning rather than bytes alone. Apply criteria-based model judges and RAG checks for correctness, faithfulness, relevance, and hallucination. Use majority, consensus, average, weighted-average, or median policies while retaining every judge's reasoning. Put cheap deterministic checks before slower execution or model-backed tiers with `CascadedJury`. Distinguish `ABSTAIN` and evaluator `ERROR` from a real negative finding instead of collapsing everything into a score. ## Bring the same evaluation policy to each runtime Spring AI, LangChain4j, Koog, and AgentClient each have their own execution model. Each Agent Judge bridge converts framework output into a `JudgmentContext`, after which the same judges and juries evaluate it. Adapters preserve native output, model identity, token usage, timing, and tool evidence in a shared `JudgmentContext`. The judges and jury policy above that boundary do not change when the producing runtime does. | Runtime | Evaluation entry point | | ------------------------------ | ---------------------- | | Spring AI | `SpringAiEvaluator` | | LangChain4j | `LangChain4jEvaluator` | | Koog | `KoogEvaluator` | | CLI agents through AgentClient | `AgentClientEvaluator` | ## Install 0.15 ```xml theme={null} io.github.markpollack agent-judge-core 0.15.1 ``` Add only the family and bridge modules that your application uses. ## Learn with executable examples The [Agent Judge Tutorial](https://github.com/markpollack/agent-judge-tutorial) progresses from one deterministic acceptance check to composed judges, voting and cascaded juries, custom and model-backed judges, and framework bridges. Every example runs without credentials. Add Agent Judge 0.15 and run the first evaluation Run the evaluation patterns as maintained Java projects Understand status-first outcomes and aggregation See why evidence, requirements, cascades, and independent checks shape the design ## Resources * [Source](https://github.com/markpollack/agent-judge) * [Tutorial source](https://github.com/markpollack/agent-judge-tutorial) * [0.15 release notes](https://github.com/markpollack/agent-judge/blob/v0.15.0/RELEASE_NOTES_0.15.md) * [0.14 migration guide](https://github.com/markpollack/agent-judge/blob/main/consumer-handoff-normalized-judgment.md) ## License The current Agent Judge source tree uses project-specific Business Source License 1.1 terms. Read the repository's [root `LICENSE`](https://github.com/markpollack/agent-judge/blob/main/LICENSE) for the exact grant, restrictions, change date, and change license; earlier published releases remain governed by the license distributed with those releases. # Agent Memory Source: https://lab.pollack.ai/projects/agent-memory Progressive memory management for Spring AI — from context compaction to autonomous memory control **[What's New →](/docs/agent-memory/whats-new)** **Latest — 0.5.1.** **0.4.0:** first Business Source License 1.1 release. Jackson floors raised for standalone consumers, an aggregate CycloneDX SBOM published on the parent artifact, hosted OWASP/NVD scanning removed in favour of a local offline procedure, and the operating boundary documented. Tier-1 compaction behaviour is unchanged from 0.1.0. ## Overview Agent Memory gives AI agents a way to manage conversational context. Without memory management, every prior tool result is re-sent every turn — on long tasks, context fills with stale noise, costs climb, and the model loses focus. Agent Memory stores accumulated learnings on the filesystem, injects a token-budgeted subset into each request, and summarizes older entries with a cheaper model once the uncompacted set crosses a threshold. The library starts with context compaction and progressively adds structured retrieval, reflection, and autonomous memory management — eventually reaching MemGPT-level capabilities. Each tier is independently useful. **Tier 1 is what ships today; tiers 2–4 are planned.** Agent Memory ships as a Spring AI `BaseAdvisor` — plug it into any `ChatClient` pipeline with one line: ```java theme={null} var memoryStore = new FileSystemMemoryStore(Path.of(".memory")); var advisor = CompactionMemoryAdvisor.builder(memoryStore) .compactionChatClient(ChatClient.create(haikuModel)) .memoryTokenBudget(8192) .compactionRatio(0.75) .build(); ChatClient agent = ChatClient.builder(chatModel) .defaultAdvisors(advisor) .build(); ``` On each request, the advisor retrieves accumulated learnings (within the token budget) and injects them into the system message. After each response, it appends the assistant's output to the store. When uncompacted entries exceed `budget × ratio`, compaction summarizes them via a cheap model and replaces them with dense summaries. ## Compatibility | Property | Value | | --------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Java | 17 | | Spring AI | 2.0.1 | | Maturity | Pre-1.0 library; Tier-1 compaction shipping | | License | **0.4.0 and later: Business Source License 1.1.** 0.3.0 and earlier remain Apache License 2.0. Published tags and artifacts are not retroactively relicensed. | ## Operating boundary The 0.4.0 filesystem store is **local, plaintext, and single-writer**. Read this before putting it in front of anything shared. * Memory is written to the local filesystem in clear text. It is not encrypted and is not shared storage. * There is no locking, no atomic index replacement, and no crash recovery. A second concurrent writer, or a crash during an index write, can corrupt or truncate `_index.json` — the only source of truth for the index. * Stored memory is injected verbatim into the model's system prompt, so only trusted content should be written to it. * Multi-process or concurrent-writer use requires external coordination. Locking, atomic index replacement, and crash recovery are backlog items, not shipped functionality. ## How Compaction Works When accumulated context exceeds a token budget, older entries are summarized by a cheap model (e.g., Haiku) and replaced with a compact summary. The agent continues with dense, relevant context instead of an ever-growing prompt. Two parameters control it: | Parameter | Default | Description | | ------------------- | ------- | -------------------------------------------- | | `memoryTokenBudget` | 8,192 | Max tokens of memory included in each prompt | | `compactionRatio` | 0.75 | Fraction of budget that triggers compaction | Token estimation is a characters÷4 approximation, not a tokenizer. ## Research background The measurements below come from [wiggum-memory](https://github.com/markpollack/wiggum-memory), the research project Agent Memory was extracted from. They were produced by that project's `RalphMemoryAdvisor`, **not** by the `CompactionMemoryAdvisor` published here, and this repository ships **no live-model benchmark** of its own. Treat them as the origin story for the design, not as performance claims for the library. ### One research run on a 12-story PRD A single research run (n=1) compared unbounded memory against token-budgeted retrieval plus compaction on a 12-story e-commerce PRD, with **Anthropic Haiku 4.5 on both sides** and the compaction step also using Haiku 4.5. **Measured configuration: `memoryTokenBudget` 4,096, `compactionRatio` 0.5.** The 8,192 / 0.75 defaults shown above were a recommendation for unstructured conversation and were never measured on this suite. | Metric | Unbounded memory | Budgeted + compaction | | -------------- | ---------------- | --------------------- | | Stories passed | 9/12 | 11/12 | | Total tokens | 56,876 | 40,152 | | Estimated cost | \$0.34 | \$0.24 | Caveats that belong with the numbers: * **n=1.** In an earlier run of the same suite at a 2,048 budget, unbounded memory scored **11/12** and the budgeted configuration scored **7/12**. The 9/12 figure above is one draw from a noisy distribution, and the write-up itself attributes the difference to ordinary LLM variability. * **Pass/fail is self-reported** by the run's own judge, not an independent evaluation. * **Cost is an estimate** derived at a \$6/MTok rate, not a billed amount. * The same write-up found that a 2,048 budget destroyed critical details (table names, endpoint signatures, auth token formats) and caused a five-story failure streak. Budget selection dominates the result. Token growth without compaction was linear (\~800 tokens/story, reaching 9,000+ by story 12). With compaction it plateaued around 4,600 tokens after the first compaction cycle. ### A related experiment in Loopy — different implementation, different models [Loopy](/projects/loopy) is a separate Spring AI agent CLI. Its `AgentLoopAdvisor` applies a related idea — threshold-based triggering with model summarization — but operates on **conversation messages** rather than accumulated learnings. It is a different implementation and is not this library. A code-coverage experiment there recorded: | Configuration | Model | Compaction threshold | Input tokens | Cost | Outcome | | ------------- | ------ | -------------------- | ------------ | ------ | ------------ | | Loopy | Haiku | none | 18,336,594 | \$2.55 | Failed | | Loopy | Sonnet | 0.5 | — | \$5.06 | Cost cap hit | | Loopy | Sonnet | 0.3 | 854,353 | \$2.72 | Passed | **The model differs across the rows**, so this is not a controlled comparison: the failing unbounded run used Haiku and the passing compacted run used Sonnet. No quality measurement was held constant, and the run that logged far fewer input tokens cost slightly *more* in dollars. The honest reading is narrow — in that experiment, compacting earlier was what let the run finish inside its cost cap. It is not evidence that this library reduces token use at equal quality. ## Roadmap | Tier | Name | Status | Description | | ---- | -------------- | ------------ | ------------------------------------------------------------------------------------------------------------------- | | 1 | **Compaction** | **Shipping** | Token-budgeted retrieval + LLM summarization | | 2 | Structured | Planned | Categorized memory with selective retrieval and per-category retention policies | | 3 | Reflective | Planned | Importance scoring + periodic reflection synthesis ([Generative Agents](https://arxiv.org/abs/2304.03442) pattern) | | 4 | Autonomous | Planned | Agent-controlled memory via tools — virtual context management ([MemGPT](https://arxiv.org/abs/2310.08560) pattern) | ## Modules | Module | Description | | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `memory-core` | `MemoryStore` interface, `FileSystemMemoryStore`, `ProgressFileMemoryStore`, `MemoryEntry`, `MemoryIndex`, `MemoryCompactor`, `TokenEstimator` | | `memory-advisor` | `CompactionMemoryAdvisor` — Spring AI `BaseAdvisor` for ChatClient integration | The on-disk shape — `iterations/`, `patterns/`, `_index.json`, or a single `progress.txt` — is unversioned and carries no forward-compatibility guarantee. ## Quick Links Source code (0.5.1 on Maven Central) ## Origin Extracted from [wiggum-memory](https://github.com/markpollack/wiggum-memory) — a research project that explored the [Ralph Wiggum pattern](https://ghuntley.com/ralph/) for context management in AI agent loops. The memory subsystem proved to be the most broadly useful component, so it was promoted to a standalone library as part of the AgentWorks stack. # Agent Sandbox Source: https://lab.pollack.ai/projects/agent-sandbox Synchronous command execution and workspace files across Local, Docker, and E2B backends **[What's New →](/docs/agent-sandbox/whats-new)** This project has moved from the `spring-ai-community` GitHub organization to `markpollack`. New releases are published under the Maven groupId `io.github.markpollack`, and Java packages now use the `io.github.markpollack` namespace. If you previously used `org.springaicommunity`, update your dependency coordinates and imports to the current values shown below. Agent Sandbox provides one synchronous command and workspace-file contract across multiple backends. The shared `exec` and file behavior is tested by the `AbstractSandboxTCK`. Backend-specific or unsupported capabilities, including interactive execution, are not interchangeable. ## Backends Process and workspace convenience via zt-exec. It provides no security isolation and is for development and trusted commands. A convenient local Docker backend using Testcontainers and an image you select. It is not a hardened multi-tenant execution service. Remote Firecracker microVM execution via E2B. This backend requires an E2B account and API key. `DockerSandbox` has privileged access to a trusted, root-equivalent local Docker daemon. Container isolation alone is not a complete hostile-workload security boundary; add the kernel, user, and network controls required by your threat model. ## Core API ```java theme={null} try (Sandbox sandbox = LocalSandbox.builder() .tempDirectory("test-") .build()) { // Execute commands ExecResult result = sandbox.exec(ExecSpec.of("mvn", "test")); if (result.success()) { System.out.println(result.stdout()); } // File operations sandbox.files() .create("pom.xml", pomContent) .create("src/main/java/App.java", code) .and() // return to Sandbox .exec(ExecSpec.of("mvn", "compile")); } ``` ## Module Structure | Module | Backend | Dependencies | | ---------------------- | --------------- | ------------------- | | `agent-sandbox-core` | `LocalSandbox` | zt-exec | | `agent-sandbox-docker` | `DockerSandbox` | Testcontainers | | `agent-sandbox-e2b` | `E2BSandbox` | jackson, awaitility | ## Maven The currently released version is **0.10.2**: ```xml theme={null} io.github.markpollack agent-sandbox-core 0.10.2 ``` Use `agent-sandbox-docker` or `agent-sandbox-e2b` for those backends. In version 0.10.0, `DockerSandbox` requires an explicit caller-selected image: the no-argument constructor is removed, and `DockerSandbox.builder().build()` fails before Docker access unless `.image(...)` is set. Constructors that already take an image continue to work. The caller may use any compatible image and owns its provenance, contents, security, and update policy. ## Source Source code — Core, Docker, and E2B modules ## Used By * **[Agent Judge](/projects/agent-judge)** — `agent-judge-exec` runs command-based evaluation in sandboxes # Agent Skills Source: https://lab.pollack.ai/projects/agent-skills Curated domain knowledge modules — SkillsJars that make agents smarter without prompt engineering This is a Spring AI Community project (`spring-ai-community/spring-testing-skills`) and is not part of the AgentWorks platform. It uses its own repository and Maven coordinates (`org.springaicommunity`). Skills are curated knowledge packages that teach agents how to do specific things well. Each skill is a structured Markdown file with routing rules and reference patterns. Agents discover skills at startup and load full content only when relevant — no tokens wasted on unused knowledge. Skills follow the [SkillsJars](https://skillsjars.io) specification and work in any agentic CLI — Loopy, Claude Code, and 40+ others. ## Spring Testing Skills The first production-ready SkillsJars collection — 7 skills that teach agents how to test Spring applications. | Skill | Domain | | ------------------------------- | --------------------------------------------------------------------------------- | | **spring-testing-router** | Meta-skill — routes to the right domain skill based on the task | | **spring-jpa-testing** | `@DataJpaTest`, Testcontainers, `@ServiceConnection`, Hibernate 6/7, lazy loading | | **spring-mvc-testing** | `@WebMvcTest`, `MockMvc`, `jsonPath`, request validation, `@RestControllerAdvice` | | **spring-security-testing** | `@WithMockUser`, CSRF, JWT, OAuth2, `@PreAuthorize` method security | | **spring-webflux-testing** | `@WebFluxTest`, `WebTestClient`, `StepVerifier`, virtual time, SSE | | **spring-websocket-testing** | STOMP/WebSocket, `WebSocketStompClient`, `BlockingQueue` pattern | | **spring-testing-fundamentals** | AssertJ, BDDMockito, `ArgumentCaptor`, context caching, anti-patterns | ### Skill Structure Each skill has a `SKILL.md` with metadata and routing rules, plus a `references/` directory with detailed patterns: ``` skills/spring-jpa-testing/ ├── SKILL.md # Metadata + routing rules └── references/ ├── testcontainers.md # Detailed patterns ├── transactional-tests.md ├── lazy-loading-tests.md └── hibernate-6-migration.md ``` ### Install ```bash theme={null} git clone https://github.com/spring-ai-community/spring-testing-skills.git cd spring-testing-skills ./mvnw package mvn skillsjars:extract -Ddir=~/.claude/skills ``` Skills are extracted to `~/.claude/skills/` where any agentic CLI discovers them automatically. ## Evidence In the [Code Coverage v2](/experiments/code-coverage-v2) experiment, adding testing skills: * Reduced JAR\_INSPECT tool calls from **11.0% → 1.0%** — the agent stopped inspecting dependencies because the skills already provided the knowledge * Reduced expected steps by **\~25%** compared to baseline **Updated 2026-08-27 — the absolute figures are withdrawn.** This read *"25% (224 vs 301)."* The `224` reproduces exactly from a chain that grouped by item rather than by run: v2 ran 2–3 runs against a single item, so every run of a variant was concatenated into one sequence, losing all but one absorption event and inflating expected steps about threefold (defect **CT6**). Recomputed with trajectories separated: **74.7 vs \~101–109**, depending on which baseline arm is meant — the page does not say, which is its own defect. **The \~25% survives** because both arms were inflated by the same factor. The absolute numbers do not. See the [v2 experiment page](/experiments/code-coverage-v2) and `agent-control-theory` `d18fc50`. This validates the thesis: curated knowledge improves agent quality more than model upgrades. ## Creating Custom Skills Any Markdown file with YAML frontmatter can be a skill: ```markdown theme={null} --- name: my-team-conventions description: Coding conventions for our Spring Boot services --- # Instructions When working on this codebase: - Use constructor injection, never field injection - All REST endpoints return ProblemDetail for errors (RFC 9457) - Tests use @WebMvcTest with MockMvc, not @SpringBootTest ``` Place in `.claude/skills/my-conventions/SKILL.md` (project) or `~/.claude/skills/my-conventions/SKILL.md` (global). ## Documentation 7 skills with routing table and reference patterns The SkillsJars format — portable domain knowledge for any agentic CLI # Agent Workflow Source: https://lab.pollack.ai/projects/agent-workflow Build agents that work — and measure why they work. Multi-step pipelines with typed context, quality gates, and portable runtimes. **[What's New →](/docs/agent-workflow/whats-new)** > **Current release: 0.12.1.** > > **What's new in 0.10.0:** Upgraded to Spring AI 2.0.0 GA, which clears CVE-2026-41712. Builds on 0.9.0 — `AgentCallback.onQuestion` now takes a tool-agnostic Question/Option callback record (no longer leaking a tool-library type), and judge integration moved to the `io.github.markpollack` namespace — plus 0.8.0 trace capture (`AgentClientStep` propagates trace file paths into the workflow journal) and 0.7.0 `ManagedAgentStep` (run steps in Anthropic's hosted sandbox). ## Overview Agent Workflow helps you build agents that really work — understand *why* they work, then improve them in a controlled, experimental manner. You compose **steps** into **workflows**, each step doing one thing: call an LLM, run a function, invoke an external agent. Quality gates evaluate output at each stage. Every step transition is traced, feeding [Agent Journal](/projects/agent-journal) for behavioral analysis — so you can answer: *does the agent need better real-time steering? What knowledge is it missing to achieve its goal? Which steps should be deterministic instead of LLM-driven? What new tools should be built?* The philosophy follows what Stripe learned building [Minions](https://arxiv.org/abs/2402.15678) at scale: *"The model does not run the system. The system runs the model."* A fluent DSL makes workflows easy to define — branching, loops, parallel execution, LLM-driven routing, error recovery. Steps exchange data through typed **context**. The workflow compiles to a **graph intermediate representation** that separates definition from execution, enabling portable runtimes without changing workflow code. ```java theme={null} Workflow.define("pr-review") .step(fetchDiff) .then(analyzeDiff) .gate(new JudgeGate(jury, 0.8)) .onPass(postComment) .onFail(revise) .end() .run(event); ``` ## Core Concepts **Steps** are the building blocks. Each step takes input, does work, and produces output. Steps can be: * **Deterministic** — a Java function (GitHub API call, string formatting, file parsing) * **Single LLM call** — `ChatClientStep` wraps a [Spring AI](https://spring.io/projects/spring-ai) `ChatClient` call * **Agentic CLI tools** — `ClaudeStep` uses the [Claude Agent SDK](/projects/claude-agent-sdk) for full multi-turn agent sessions with deep tracing. `AgentClientStep` wraps other agentic CLI tools — Google Gemini, OpenAI Codex, Amazon Q — via [Agent Client](/projects/agent-client), giving you a unified interface * **Hosted agents** — `ManagedAgentStep` delegates to [Anthropic Managed Agents](https://docs.anthropic.com/en/api/overview), running steps in Anthropic's cloud sandbox with full tool access. `A2AStep` delegates via the A2A protocol to any remote agent A `ClaudeStep` or `AgentClientStep` isn't a single API call — it runs a complete agentic loop internally (dozens of tool calls, minutes of execution) and returns a typed result. The workflow sees it as one step. **Context** threads through every step. Steps read input parameters by key, do their work, and write output parameters back. Downstream steps pick up what upstream steps produced — all type-safe via `ContextKey`. **The graph** means the workflow definition is pure data — nodes and edges, not opaque lambdas. This enables: * **Portable runtimes** — the graph decouples definition from execution. Ships with `LocalStepRunner` (in-process, zero overhead), `CheckpointingStepRunner` (JDBC crash recovery via `workflow-batch`), and `TemporalStepRunner` (distributed durable execution via `workflow-temporal`) — same workflow code, swap a single `@Bean` * **Tracing** — every step transition is recorded for observability and behavioral analysis via [Agent Journal](/projects/agent-journal) * **Steering** (planned) — runtime hooks that intercept before/after steps to enforce constraints, redirect behavior, or inject guidance. Deterministic or LLM-powered. Integrates with [Spring AI](https://spring.io/projects/spring-ai) advisors and the [Claude Agent SDK](/projects/claude-agent-sdk) hook system * **Inspection** — the graph is pure data (nodes + edges), not opaque lambdas ## Start Here From a single step to a supervised agent pipeline — 8 progressive examples Clone and run all 13 examples with real LLM calls ## Documentation Steps, context, portable runtimes, first workflow 10+ composable patterns with code @Agent, AgentHandler, exception handling, registry Crash recovery, checkpointing, Temporal integration 4 patterns for getting data into steps Step, AgentContext, Gate, WorkflowGraph, StepRunner ## Why Deterministic Steps Matter The biggest insight from running real agent experiments: the AI shouldn't do everything. This is the pattern Stripe describes in their [Minions](https://stripe.dev/blog/minions-stripes-one-shot-end-to-end-coding-agents) system, now shipping 1,300 PRs a week: *the model does not run the system — the system runs the model.* A real [PR merge workflow](https://github.com/markpollack/agentworks-pr-review) illustrates the point. The pipeline has 8 steps: ``` checkoutPR → formatCode → collectContext → compile → squash → rebase → resolveConflicts → review ``` Six steps are deterministic — git operations, Java formatting, GitHub API calls, Maven compile. Only two need LLM reasoning: resolving merge conflicts and reviewing the diff. The deterministic steps are free, fast, and perfectly reliable. The LLM steps are expensive and variable. By minimizing what the LLM needs to do, you reduce cost, increase reliability, and make the whole pipeline easier to debug. This isn't obvious until you measure it. In our [code coverage experiments](/experiments/code-coverage-v2), adding a deterministic pre-analysis step cut agent steps by 27%. The agent still explored the codebase — but it read source files instead of decompiling JARs. Same attention budget, better allocation. ## Resources Source code (0.12.1 on Maven Central) 13 runnable examples — validated with real LLM calls ## Used In * [Code Coverage v1](/experiments/code-coverage-v1) — Agent execution engine for all 9 variants * [Code Coverage v2](/experiments/code-coverage-v2) — Agent execution with skills injection * [Issue Classification](/experiments/issue-classification) — SWE-bench agent runner # AgentWorks BOM Source: https://lab.pollack.ai/projects/agentworks-bom Bill of Materials for coordinated version management across all AgentWorks projects **[What's New →](/docs/agentworks-bom/whats-new)** ## Overview The AgentWorks BOM pins compatible versions of every library in the agent engineering stack. Import it once and drop version tags from your dependency declarations. ## Maven ```xml theme={null} io.github.markpollack agentworks-bom 1.18.0 pom import io.github.markpollack experiment-core ``` ## What's new in 1.18.0 * **1.18.0:** a currency repin, and the point at which the BOM caught up with its members — every managed coordinate is now that project's current release: Agent Client **0.29.3**, Agent Journal **1.8.2**, Agent Judge **0.15.1**, Agent Workflow **0.12.1**, the acp-java family **0.16.1**, Agent Hooks **0.8.2**, Agent Sandbox **0.10.2**, Agent Experiment **0.7.1**, Agent Bench **0.6.1**, Agent Memory **0.5.1**, Claude Agent SDK **1.5.1**. The work behind 1.17.0 and 1.18.0 was predominantly security and SBOM hardening, not features. Managed members grew 46 → 72 as more published modules came under management (Agent Client 12 → 28, Agent Judge 4 → 10, Agent Journal 3 → 6). * **1.16.0:** the broadest repin since the org migration — Agent Client **0.28.0**, Agent Journal **1.7.0**, Agent Hooks **0.7.0**, Agent Judge **0.15.0**, Agent Sandbox **0.10.0**, Agent Bench **0.6.0**, Agent Experiment **0.6.0**, Agent Memory **0.4.0**, Claude Agent SDK **1.5.0**, and the acp-java family **0.15.0**. `loopy` is no longer a managed member; depend on it directly. * **1.15.0:** repins the Agent Client family to **0.25.0** — fixes Spring Boot auto-configuration for all providers. The org migration had left the auto-config registration files (`.imports`/`spring.factories`) pointing at the old `org.springaicommunity` package, so the starters failed to auto-configure (`ClassNotFoundException`) at boot. All other members unchanged. * **1.14.0:** repins the Agent Client family to **0.24.0** — a compatibility fix so `agent-claude` builds against agent-journal 1.6.0. journal 1.5.0 had relocated `TraceContentMode`, which broke Claude trace-wiring for Spring Boot consumers on 1.12.0/1.13.0 (`NoClassDefFoundError`). All other members unchanged. * **1.13.0:** repins Agent Journal to **1.6.0** — first-class journal-capture primitives (slice 1), a cost-metering correction (the headline `LLMCallEvent.tokenUsage` is now the cost-bearing Σ-per-turn aggregate, fixing a \~2× long-run under-count), and a per-file schema-version header on the jsonl logs. Additive on the frozen capture contract; a 1.5.0 consumer keeps working. * **1.12.0:** repins Agent Journal to **1.5.0** and adds the new **`gemini-cli-capture`** managed member — agent runs from Claude *and* Gemini now capture into one portable trace + cost format. * **1.9.0 was the security cut:** converged on Spring AI 2.0.0 GA and cleared four CVEs — Spring Boot CVE-2026-40976 (CRITICAL) and CVE-2026-40973, Spring AI CVE-2026-41712, and Jackson CVE-2026-29062. Added `tools.jackson:jackson-bom` 3.1.4 and `reactor-core` 3.8.6 to converge transitive dependencies, all verified by the `bom-verification` gate. * **1.10.0 / 1.11.0:** repinned the source-fixed members to lock in the CVE remediation (claude-code-sdk 1.4.0, workflow 0.10.0, Spring AI BOM 2.0.0 backstop). * **1.8.0:** bumped the acp-java family to 0.14.0, clearing the deferred 0.13.0 cut. ## Managed Artifacts ### Agent Engineering (io.github.markpollack) | Artifact | Version | Project | | ---------------------------------------- | ------- | ---------------------------------------------- | | `agent-hooks-core` | 0.8.2 | [Agent Hooks](/projects/agent-hooks) | | `agent-hooks-spring` | 0.8.2 | [Agent Hooks](/projects/agent-hooks) | | `agent-hooks-claude` | 0.8.2 | [Agent Hooks](/projects/agent-hooks) | | `agent-hooks-gemini` | 0.8.2 | [Agent Hooks](/projects/agent-hooks) | | `journal-core` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `claude-code-capture` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `gemini-cli-capture` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `antigravity-cli-capture` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `codex-cli-capture` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `grok-cli-capture` | 1.8.2 | [Agent Journal](/projects/agent-journal) | | `workflow-api` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-core` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-tools` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-flows` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-agents` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-batch` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-temporal` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `workflow-journal` | 0.12.1 | [Agent Workflow](/projects/agent-workflow) | | `experiment-core` | 0.7.1 | [Agent Experiment](/projects/agent-experiment) | | `experiment-claude` | 0.7.1 | [Agent Experiment](/projects/agent-experiment) | | `experiment-workflow` | 0.7.1 | [Agent Experiment](/projects/agent-experiment) | | `memory-core` | 0.5.1 | [Agent Memory](/projects/agent-memory) | | `memory-advisor` | 0.5.1 | [Agent Memory](/projects/agent-memory) | | `claude-code-sdk` | 1.5.1 | [Claude Agent SDK](/projects/claude-agent-sdk) | | `agent-bench-core` | 0.6.1 | [Agent Bench](/projects/agent-bench) | | `agent-bench-agents` | 0.6.1 | [Agent Bench](/projects/agent-bench) | | `agent-judge-core` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-llm` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-exec` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-file` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-agent-client` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-ai-core` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-koog` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-langchain4j` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-rag` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-judge-spring-ai` | 0.15.1 | [Agent Judge](/projects/agent-judge) | | `agent-sandbox-core` | 0.10.2 | [Agent Sandbox](/projects/agent-sandbox) | | `agent-sandbox-docker` | 0.10.2 | [Agent Sandbox](/projects/agent-sandbox) | | `agent-sandbox-e2b` | 0.10.2 | [Agent Sandbox](/projects/agent-sandbox) | | `agent-client-core` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-client-spring-boot-autoconfigure` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-model` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-claude` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-gemini` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-launcher` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-qwen-code` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-amazon-q` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-amp` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-antigravity` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-codex` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-grok` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-swe` | 0.29.3 | [Agent Client](/projects/agent-client) | | `amazon-q-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `amp-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `antigravity-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `codex-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `gemini-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `grok-cli-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `swe-agent-sdk` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-claude` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-gemini` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-amp` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-codex` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-amazon-q` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-qwen-code` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-antigravity` | 0.29.3 | [Agent Client](/projects/agent-client) | | `agent-starter-grok` | 0.29.3 | [Agent Client](/projects/agent-client) | ### Agent Client Protocol (com.agentclientprotocol) | Artifact | Version | Project | | --------------------- | ------- | -------------------------------------- | | `acp-annotations` | 0.16.1 | [ACP Java SDK](/projects/acp-java-sdk) | | `acp-core` | 0.16.1 | [ACP Java SDK](/projects/acp-java-sdk) | | `acp-agent-support` | 0.16.1 | [ACP Java SDK](/projects/acp-java-sdk) | | `acp-test` | 0.16.1 | [ACP Java SDK](/projects/acp-java-sdk) | | `acp-websocket-jetty` | 0.16.1 | [ACP Java SDK](/projects/acp-java-sdk) | ## Source BOM source and release coordination # Bud Source: https://lab.pollack.ai/projects/bud ACP agent for Spring Boot development — proven patterns as a starting point, AI to adapt them **Early Stage** — Bud is functional but under active development. The scaffolding and modification tools work end-to-end, but CLI provider portability (beyond Claude Code) and code generation into existing projects are in progress for v0.5.0. ## Overview Bud is a Spring Boot development agent that lives in your IDE. Select Bud and say "create a REST app" or "add JPA to my project." It understands what you're building and does the work — creating new projects, modifying existing ones, managing dependencies — all through conversation. **Zero API keys. Your CLI, your rules.** Bud makes no direct LLM calls and carries no model credentials. It delegates to whatever agentic CLI your enterprise has approved — Claude Code, Gemini CLI, Codex. Your IT department controls the AI provider, the billing, and the compliance story. Bud is the agent; your CLI is the brain. **ACP-native.** Bud is an ACP agent — any IDE that supports the Agent Communication Protocol can host it. JetBrains, Zed, VS Code, and any future ACP-compatible editor. ## How It Works Bud ships with 9 production-quality reference projects (REST, JPA, Security, Batch, and more) that follow Spring Boot conventions — constructor injection, slice tests, proper layering. When your request fits a known pattern, Bud uses that working code directly. When it doesn't, the LLM on your approved CLI classifies your intent and generates domain-specific code that follows the same conventions. * **23 deterministic tools** — `create_project`, `add_dependency`, `add_actuator`, `analyze_project`, and more. No LLM calls — pure code operations. * **9 reference projects** — Production-quality starting points, not stubs. REST, JPA, Security, Actuator, Batch, Scheduling, Thymeleaf, Spring AI, Minimal. * **Modify existing projects** — Add dependencies, apply recipes, analyze project structure through conversation. ## Architecture ``` IDE (JetBrains / Zed / VS Code) └─ ACP Client └─ Bud ACP Agent └─ AgentClient → Your Approved CLI (Claude Code, Gemini CLI, Codex...) └─ Bud MCP Server (23 deterministic tools) └─ bud-core (pure Java library) ``` ## Availability Bud is in active private development and is **not publicly distributed today**. The repository is private, and there is no public download or registry package. If you would like a walkthrough, early access, or to talk about evaluating Bud in your environment, get in touch: **[mark@pollack.ai](mailto:mark@pollack.ai)** # Claude Agent SDK (Java) Source: https://lab.pollack.ai/projects/claude-agent-sdk Java SDK for Claude Code CLI integration — three-API architecture, sessions, MCP, multi-agent orchestration **[What's New →](/docs/claude-agent-sdk/whats-new)** This project has moved from the `spring-ai-community` GitHub organization to `markpollack`. New releases are published under the Maven groupId `io.github.markpollack`, and Java packages now use the `io.github.markpollack` namespace. If you previously used `org.springaicommunity`, update your dependency coordinates and imports to the current values shown below. Java SDK for programmatic access to Claude Code. Three APIs at different abstraction levels: `Query` (one-liner fire-and-forget), `ClaudeSyncClient` (blocking sessions), and `ClaudeAsyncClient` (reactive with Project Reactor). Pure Java — no native dependencies, and no Spring dependency either: the SDK module depends only on zt-exec, Jackson, Reactor, SLF4J and the MCP Java SDK. Spring Boot auto-configuration for Claude lives in [Agent Client](/projects/agent-client), not here. This is the SDK that powers `ClaudeStep` in [Agent Workflow](/projects/agent-workflow) — each step runs a full multi-turn Claude session internally. **Current release: 1.5.1.** **1.5.0:** A security, packaging and behaviour-correction release: it raises the Jackson floors so they reach standalone consumers, embeds the Apache licence in every archive, publishes a CycloneDX 1.6 SBOM, and stops a no-argument `connect()` from sending a prompt of its own. It follows 1.4.0, which bumped the MCP SDK from 0.15.0 to 2.0.0 (mcp-core 2.0.0), clearing CVE-2026-35568. See [What's New →](/docs/claude-agent-sdk/whats-new). ## Using it ```xml theme={null} io.github.markpollack claude-code-sdk 1.5.1 ``` **Java 21 or later.** Every published artifact is Java 21 bytecode (class-file major 65); there has never been a Java 17 build, and a Java 17 runtime cannot load the SDK. **Upgrade from 1.4.0.** If you use 1.4.0 without importing a BOM that manages Jackson, your runtime closure resolves Jackson 2.21.2 and Jackson 3.0.3, which carry 23 known vulnerabilities (8 HIGH). 1.5.0 declares the Jackson 2.21.6 / 3.1.6 floors directly on the published module, so the same no-BOM consumer resolves zero findings. 1.4.0 stays published and stays exposed — Maven Central is immutable, and nothing repairs it retroactively. ## How it operates The SDK does not call the Anthropic API. It **spawns the Claude Code CLI as a child process** and speaks the stream-JSON control protocol to it over stdin and stdout. That shapes what you need to know: * The `claude` executable must be installed and authenticated on the machine that runs your code. The SDK discovers it on `PATH` and in the usual install locations. * `ANTHROPIC_API_KEY` is passed through to the child process when set. Environment inheritance is otherwise a whitelist, not a copy of your whole environment. * Every session consumes real model usage and is billed. That includes the integration test suite in the SDK repository. A no-argument `connect()` sends nothing on its own: from 1.5.0 it starts and initialises the session and leaves the first turn to your `query(...)`. Before 1.5.0 it substituted a literal `"Hello"` and billed that turn. * The child process, and its descendants, are terminated on close. ## Licence Apache License 2.0 — see [the repository LICENSE](https://github.com/markpollack/claude-agent-sdk-java/blob/main/LICENSE). This SDK is a supporting public project and is not part of the AgentWorks BSL set. ## Tutorials 23 progressive modules from fundamentals to multi-agent orchestration: Query API, sync client, async client, message types CLI options, tool permissions, permission modes, structured outputs Multi-turn conversations, session resume, session fork Permission callbacks, pre/post tool-use hooks, interrupt handling External MCP servers, multiple servers, MCP with hooks Subagent definitions, parallel subagents, orchestrator pattern ## Source SDK implementation 23 runnable tutorial modules ## Role in the Lab * **[Agent Workflow](/projects/agent-workflow)** uses `ClaudeStep` which wraps `ClaudeSyncClient` — each workflow step runs a full multi-turn Claude session * **[Agent Client](/projects/agent-client)** uses the SDK for Claude Code integration with Spring Boot lifecycle and sandbox isolation * All [experiments](/experiments/index) invoke Claude through this SDK via the experiment driver's `ClaudeSdkInvoker` # Projects Source: https://lab.pollack.ai/projects/index Tools for building, evaluating, and observing AI agents on the JVM The lab produces a focused set of projects that work together as an agent engineering stack — for building, evaluating, and understanding AI agents on the JVM. ## AgentWorks Core `agent-*` libraries that compose into a complete agent engineering stack. Use the [AgentWorks BOM](/projects/agentworks-bom) for coordinated version management. Compose multi-step agentic pipelines — steps, typed context, branching, loops, quality gates, parallel execution. Compiles to a graph IR with portable runtimes. Autonomous CLI agent integrations — a unified framework actively verified with Claude Code, Codex, and Gemini CLI. Other adapters remain experimental. Portable MCP servers, auto-configuration, and reasoning effort (`low`/`medium`/`high`) across providers, with provider-native ranges when finer control is needed. Agent-agnostic evaluation with deterministic, command, and LLM judges. Jury system with voting strategies. Zero cost for T0-T1 checks — LLM fires only when cheaper tiers pass. Benchmarking suite for Java-centric AI agents on enterprise dev tasks — issue triage, PR review, coverage, compliance. Agent-agnostic with provide/grade separation. End-to-end experiment driver — datasets, runner, jury scoring, variant comparison, sessions, and sweeps. The execution backbone of every lab experiment. Progressive memory management — context compaction, token-budgeted retrieval, LLM summarization. Ships as a Spring AI BaseAdvisor; Tier-1 compaction today, tiers 2-4 planned. Portable hook API for steering agent behavior at the tool-call boundary — deterministic safety, observability, and input modification that works across any runtime. Behavioral trace capture — every tool call, state transition, and decision point. Feeds Markov fingerprinting and cross-variant behavioral comparison. Isolated command execution — local, Docker, or E2B cloud backends behind a unified API. Safe execution for untrusted agent code. Claude Code-inspired tools for Spring AI agents — file I/O, shell, search, web, subagent orchestration. Curated domain knowledge modules (SkillsJars) — teach agents how to test, review, and build Spring applications. ## SDKs & Protocols Foundational SDKs for agent communication and CLI integration. Java SDK for Claude Code CLI — three-API architecture, 23-part tutorial, Spring Boot auto-configuration Agent Communication Protocol — build agents, consume agents, integrate with Zed, JetBrains, and VS Code ## Older / Inactive Projects Kept published for reference. No longer under active development A small agentic coding CLI for Java developers — an agent loop, a terminal UI, and multi-provider support, with nothing else bolted on. # Loopy Source: https://lab.pollack.ai/projects/loopy A small agentic coding CLI for Java developers ## Overview Loopy is a small agentic coding CLI for Java developers — an agent loop, a terminal UI, and multi-provider support, built on Spring AI. It is a harness: somewhere to run an agent against a Java codebase, with curated domain skills available when you want them. Loopy is **not** the entry point to the [Forge methodology](/methodology/forge). Forge is used through its own slash commands. Loopy was an early surface for some of them; that role has moved. **Version 0.5.0** — [Maven Central](https://central.sonatype.com/search?q=g:io.github.markpollack+a:loopy) | [Download](https://github.com/markpollack/loopy/releases/latest) **0.5.0:** The distribution is repaired. Releases now publish an executable `loopy--exec.jar` as a GitHub Release asset and a plain library jar to Maven Central; previously the coordinate was a Spring Boot fat jar nothing could compile against, and no release had a downloadable asset at all. Spring AI moves to 2.0.0 GA with Spring Boot 4.1.0, and `--help` and `--version` no longer require an API key. ## Key Features * **Embedded agent loop** — Built on [Agent Workflow](/projects/agent-workflow) for reliable execution * **Domain skills** — Modular, curated capabilities (Spring Boot, testing, migration) * **Modern TUI** — Terminal user interface with real-time agent feedback * **Multi-provider** — Swap the model behind the loop without changing the harness ## Why Loopy? Most coding agents are generic. Loopy is opinionated about Java — it ships with curated knowledge about Java and Spring development, structured as skills that shape the agent's tool selection and execution strategy. The result is more reliable completions with fewer wasted tokens. ## Documentation Install and run your first agent session in 5 minutes Custom skills, subagents, tool profiles, and the programmatic API All flags, slash commands, and configuration options ## Quick Links Source code and documentation Knowledge-directed execution explained # What's New Source: https://lab.pollack.ai/whats-new Weekly release highlights across all Pollack AI Lab projects ## Week of August 16, 2026 ### Agent Client 0.29.0 **Grok, Codex, and Antigravity now return a real tool trajectory through the same facade contract Claude has always honoured.** * All three providers publish their parsed run trajectory through `AgentClientResponse.getPhaseCapture()`; live facade integration gates verify each one returns a capture containing tool uses, and offline model tests assert capture is additive rather than a replacement for the provider's terminal result * Grok reads its native `streaming-json` ACP stream, Antigravity its `stream-json`, and Codex harvests its durable rollout JSONL after execution, matching the file by session id or working directory with a bounded wait for delayed flushes * The adapters consume the released Agent Journal 1.8.0 capture modules, so capture stays provider-owned and `agent-model` and the production surface of `agent-client-core` take on no journal dependency * LOOSE-mode Codex execution is repaired: current Codex versions reject `--full-auto` after `exec`, so full-auto now maps to the global `--sandbox workspace-write` and `--ask-for-approval never` options and stays distinct from the unrestricted bypass level, which portable auto-approve no longer grants implicitly * All 29 modules are published, `agent-tck` included [Project](/projects/agent-client) | [What's New](/docs/agent-client/whats-new) | [Source](https://github.com/markpollack/agent-client) ### Agent Journal 1.8.0 **Tool-trajectory capture for three more CLIs, each parsed from its own durable stream.** * New `grok-cli-capture`, `codex-cli-capture`, and `antigravity-cli-capture` modules turn each CLI's JSONL into ordered `ToolCallEvent` and `StepCostEvent` records with stable tool-call identities * Codex capture sidesteps the outer-`exec` trap by classifying the nested `payload.input` call, assigning names such as `Search`, `Read`, `Inspect`, `Test`, `Build`, and `Git`, and falling back to `Shell` — with the raw command retained — wherever shell syntax makes the intent ambiguous * Grok's stream has no durable turn-to-tool cost join, so its real session total is kept and split evenly across captured steps; Codex and Antigravity report no monetary cost at all, and say so explicitly with `costAvailable=false` * Gemini CLI capture remains turn-level and is deliberately excluded from the multi-CLI tool-trajectory claim * Additive on the existing event and trace contracts; 488 tests pass across the seven-module reactor [Project](/projects/agent-journal) | [What's New](/docs/agent-journal/whats-new) | [Source](https://github.com/markpollack/agent-journal) ### Agent Client 0.28.0 **Two new CLI providers, both carried through the full parity TCK.** * A Grok CLI provider — `agent-grok`, `grok-cli-sdk`, `agent-starter-grok` — whose headless runs return a real JSON envelope, so session id, token usage and a genuine per-run USD cost are read rather than scraped, and a caller-supplied session UUID makes resume possible without recovering an id from output * An Antigravity provider for Google's `agy` — `agent-antigravity`, `antigravity-cli-sdk`, `agent-starter-antigravity` — which declares its working directory with `--add-dir` so the CLI cannot divert writes to a shared scratch directory while reporting success * Antigravity run success is derived from whether work was produced and nothing was refused, not from its `status` field, which reports ERROR alongside complete and correct responses; soft denials are detected from both the result envelope and stderr * `GROK` and `ANTIGRAVITY` join the provider parity TCK and pass all ten scenarios each against live CLIs, with zero skips and zero failures. CI cannot re-verify them because both CLIs authenticate interactively and cache credentials rather than reading an API key [Project](/projects/agent-client) | [What's New](/docs/agent-client/whats-new) | [Source](https://github.com/markpollack/agent-client) ### AgentWorks BOM 1.16.0 **The managed set catches up across the whole portfolio.** * Repins the Agent Client family to 0.28.0, bringing the Grok and Antigravity providers into the managed set * Repins Agent Journal 1.7.0, Agent Hooks 0.7.0, Agent Judge 0.15.0, Agent Sandbox 0.10.0, Agent Bench 0.6.0, Agent Experiment 0.6.0, Agent Memory 0.4.0, Claude Agent SDK 1.5.0, and the acp-java family 0.15.0 * Drops `loopy` from the managed set; it ships an executable distribution and is depended on directly * The BOM tracks a set verified together, so it trails the newest individual releases — Agent Client 0.29.0 and Agent Journal 1.8.0 must be declared explicitly [Project](/projects/agentworks-bom) | [What's New](/docs/agentworks-bom/whats-new) | [Source](https://github.com/markpollack/agentworks) ### ACP Java SDK 0.15.0 **Notifications arrive in order and survive a graceful close; the published closure and the license are now what they claim to be.** * Incoming notifications are serialized through a sink drained by `concatMap`, so a handler doing async work no longer observes rapid `session/update` chunks out of order (reported and fixed by @ljiro) * `closeGracefully()` waits for queued notifications to drain before tearing the session down, bounded by the session `requestTimeout`; previously the whole backlog could be discarded on close * Jackson 2.21.5 and Jetty 12.0.37 clear 17 known advisories (5 high) reaching consumers as compile-scope transitives * `LICENSE` is now the verbatim Apache 2.0 — the previous file omitted the Trademarks section and rewrote the copyright grant — and LICENSE plus NOTICE ship under `META-INF` in every artifact * Three `*IT` classes that `mvn verify` had never executed now run in CI [Project](/projects/acp-java-sdk) | [What's New](/docs/acp-java-sdk/whats-new) | [Source](https://github.com/agentclientprotocol/java-sdk) ### Agent Client 0.27.0 **Agent Client now verifies the dependency graph that published consumers actually receive.** * Direct Jackson and Log4j declarations preserve the accepted security floors after child POM flattening instead of relying on source-reactor parent management * A generated CI gate discovers every published runtime module and resolves fresh one-coordinate, no-parent, no-BOM consumer closures before accepting the build * The compatible dependency train moves to Claude SDK 1.5.0, Agent Journal and Capture 1.7.0, and Agent Sandbox Core 0.10.0 while retaining Java 21 * Current source headers and all published archive license payloads now agree with the Mark Pollack BSL distribution while retaining Apache history; immutable 0.26.0 artifacts are disclosed, not described as rebuilt [Project](/projects/agent-client) | [What's New](/docs/agent-client/whats-new) | [Source](https://github.com/markpollack/agent-client) ### Loopy 0.5.0 **Loopy's distribution is repaired: there is now an executable jar you can actually download and a library jar you can actually depend on.** * Releases publish `loopy-0.5.0-exec.jar` as a GitHub Release asset and a plain library jar to Maven Central; the coordinate was previously a Spring Boot fat jar nothing could compile against, and no release had a downloadable asset attached * `loopy` and `loopy.bat` launchers locate the jar and set the JVM flags Loopy expects, and `jbang loopy@markpollack/loopy` works for the first time * Spring AI moves to 2.0.0 GA with agent-utils 0.10.0 and workflow-core 0.10.0, alongside Spring Boot 4.1.0 — Loopy was the last member on a Spring AI milestone * `--help` and `--version` no longer require an API key, and `--version` reports the real version instead of the hardcoded `0.1.0-SNAPSHOT` every release since 0.2.0 announced [Project](/projects/loopy) | [What's New](/docs/loopy/whats-new) | [Source](https://github.com/markpollack/loopy) ### Agent Hooks 0.7.0 **Consumer-safe runtime closures, honest Java boundaries, and release-complete artifacts.** * Standalone no-BOM consumers now resolve Jackson 2.21.6 and 3.1.6 through direct nearest-wins floors, including the actual Hooks-plus-supplied-Claude-SDK 1.5.0 shape * Published POMs no longer inject snapshot or milestone repositories into downstream builds, and every distributed archive now contains the BSL 1.1 license text * The Claude adapter declares its real Java 21 requirement while core, Spring, and Gemini remain Java 17; hook-failure and observation-only steering warnings are now visible * The parent publishes one aggregate CycloneDX 1.6 SBOM, release workflows are immutably pinned, and public Central consumer closures have zero findings [Project](/projects/agent-hooks) | [What's New](/docs/agent-hooks/whats-new) | [Source](https://github.com/markpollack/agent-hooks) ### Agent Memory 0.4.0 **First BSL 1.1 release, consumer-safe Jackson resolution, a published SBOM, and an honest operating boundary.** * 0.4.0 and later are Business Source License 1.1; 0.3.0 and earlier remain Apache 2.0, and published artifacts are not retroactively relicensed * Jackson 3 core and databind are declared directly on memory-core so standalone no-BOM consumers resolve 3.1.6 by nearest-wins, with a committed CI resolution gate * The parent artifact publishes one aggregate CycloneDX 1.6 SBOM; hosted OWASP/NVD scanning is removed in favour of a local offline Trivy procedure * The store is documented as local, plaintext and single-writer, with no locking, atomic index replacement or crash recovery * Overstated research figures are removed from the project page; the surviving measurements are labelled as one wiggum-memory research run of a different implementation [Project](/projects/agent-memory) | [What's New](/docs/agent-memory/whats-new) | [Source](https://github.com/markpollack/agent-memory) ### Agent Experiment 0.6.0 **Workflow-backed experiments, first-class journal evidence, and a consumer-safe release graph.** * The new experiment-workflow module adapts typed Agent Workflow executions and step cost/token evidence to the AgentInvoker contract * AgentExperiment now owns the per-item Agent Journal lifecycle, while stored results use normalized Experiment-owned judgments and automatically read the older 0.5/Judge 0.13 format * Standalone consumers resolve Journal/Capture 1.7.0, Jackson 2.21.6, and Jackson 3.1.6 without a BOM; the verified 25-, 26-, and 57-JAR closures have zero findings * The Java 21 release passed 544 tests and publishes signed binaries, sources, Javadocs, and one aggregate CycloneDX 1.6 SBOM [Project](/projects/agent-experiment) | [What's New](/docs/agent-experiment/whats-new) | [Template](https://github.com/markpollack/agent-experiment-template) | [Source](https://github.com/markpollack/agent-experiment) ### Agent Journal 1.7.0 **CVE-clear standalone resolution, license-complete artifacts, and a published SBOM.** * Ordinary no-BOM consumers now resolve Jackson 2.21.6 and, for Claude capture's transitive Jackson 3 path, 3.1.6 through direct nearest-wins dependencies * Every binary and source JAR now carries META-INF/LICENSE, every Javadoc JAR carries resources/LICENSE, and the BSL 1.1 terms are unchanged * The parent artifact now publishes one aggregate CycloneDX 1.6 JSON SBOM covering all three modules and their shipped dependency closure * Capture-module POMs now declare their actual Java 21 runtime requirement while journal-core remains Java 17; APIs and stream schemas are unchanged [Project](/projects/agent-journal) | [What's New](/docs/agent-journal/whats-new) | [Source](https://github.com/markpollack/agent-journal) ### Agent Client 0.26.0 **A smaller, supportable Agent Client surface for the current provider generation.** * Claude Code, Codex, and Gemini CLI remain the actively verified provider set, with all three live parity jobs passing at the reviewed release candidate * The abandoned Vendir context advisor and Git-repository DSL are removed as an intentional breaking cleanup * The stale agents-runtime container build and Docker TCK are retired; Agent Client is now a Maven-only library distribution * Jackson and Log4j security pins, immutable release-workflow references, complete BSL text, and exact-SHA SBOM/security evidence prepare the repository for diligence review [Project](/projects/agent-client) | [What's New](/docs/agent-client/whats-new) | [Source](https://github.com/markpollack/agent-client) ### Agent Judge 0.14.0 **Status-first, portable evaluation that preserves what every judge and composite stage actually reported.** * The normalized Judgment model keeps PASS, FAIL, ABSTAIN, and ERROR distinct while treating score and classification as independent optional facts * Named composite attempts preserve stage identity, relation, policy, returned judgments, and code-only failure evidence without fabricating a verdict * Portable metadata, aggregation evidence, timing, and six token-usage quantities survive JSON and process boundaries * Framework bridges cover Spring AI 2.0.0, LangChain4j 1.19.0, Koog 1.1.1, and Agent Client 0.25.0; the public ten-module tutorial exercises the contract without credentials [Project](/projects/agent-judge) | [What's New](/docs/agent-judge/whats-new) | [Source](https://github.com/markpollack/agent-judge) *** ## Week of June 29, 2026 ### Agent Journal 1.6.0 **First-class capture primitives + a cost-metering correction.** Slice 1 of the cross-repo journal-capture contract ships the §4 primitives downstream repos import, alongside a fix that roughly halves a long-run cost-accounting error. * New primitives: `PhaseCapture.stepCosts()`, `JournalSteps.fromEvents()`, a production fail-loud `RunRecorder`, and per-turn usage in the immutable log * Cost-metering fix: the headline `LLMCallEvent.tokenUsage` is now the cost-bearing Σ-per-turn-by-type aggregate (incl. cache) instead of the last-`ResultMessage` snapshot, which under-counted \~2× on long runs — same field, corrected value, no schema change * A per-file `schemaVersion` header on `events.jsonl` / `analysis.jsonl` (loaders skip it) * Additive on the frozen capture contract — a 1.5.0 consumer keeps working; 482 tests green * [Project](/projects/agent-journal) | [What's New](/docs/agent-journal/whats-new) | [Source](https://github.com/markpollack/agent-journal) ### AgentWorks BOM 1.13.0 **Repins Agent Journal to 1.6.0.** A focused, journal-only bump behind the usual gate. * All three journal artifacts (`journal-core`, `claude-code-capture`, `gemini-cli-capture`) → 1.6.0; every other member unchanged * Published behind the CI-enforced `bom-verification` gate (plain / Boot 3.5 / Boot 4), re-verified from Central post-publish * [Version Table](/projects/agentworks-bom) | [Source](https://github.com/markpollack/agentworks) *** ## Week of June 15, 2026 ### Agent Journal 1.5.0 **First-class Gemini CLI capture.** The new `gemini-cli-capture` module projects Gemini runs into the same portable trace + cost schema as Claude, so one analysis layer reads both runtimes — plus per-step cost attribution. * New `gemini-cli-capture` module — cross-runtime capture into one trace + cost format * Per-step cost attribution (`StepCostEvent`, `analysis.jsonl` sidecar); portable `TraceWriter` moved into `journal-core` * [Project](/projects/agent-journal) | [What's New](/docs/agent-journal/whats-new) | [Source](https://github.com/markpollack/agent-journal) ### AgentWorks BOM 1.10.0 → 1.12.0 **CVE remediation locked in, then the Gemini capture cut.** Three BOM releases close out the Spring AI 2.0 security work and add the new journal member. * 1.10.0 / 1.11.0 (Jun 15): repin the source-fixed members to lock in the CVE remediation — claude-code-sdk 1.4.0, workflow 0.10.0, Spring AI BOM 2.0.0 backstop * 1.12.0 (Jun 17): repin Agent Journal to 1.5.0 and add `gemini-cli-capture` as a managed member — the first BOM release through the CI-gated `release.yml` (pre-publish gate + post-publish FINAL\_PROOF) * [Version Table](/projects/agentworks-bom) | [Source](https://github.com/markpollack/agentworks) *** ## Week of June 8, 2026 ### AgentWorks BOM 1.9.0 **Suite-wide security release: Spring AI 2.0 GA convergence, four CVEs cleared.** The BOM moves the suite onto Spring AI 2.0.0 GA and Spring Boot 4.0.7, and adds Jackson 3 and Reactor management so consumers resolve a vulnerability-free, converged dependency set — proven end-to-end against the published artifacts by the `bom-verification` gate. * Clears Spring Boot CVE-2026-40976 (CRITICAL) and -40973 (HIGH), Spring AI CVE-2026-41712 (HIGH), and Jackson CVE-2026-29062 (HIGH) * New managed import `tools.jackson:jackson-bom 3.1.4` — converges Spring AI 2.0's transitive Jackson 3 (its jsonschema-generator pulls `jackson-core 3.0.3`) up to a CVE-clear 3.1.4 * New managed `reactor-core 3.8.6` — reconciles claude-code-sdk's 3.8.5 with Spring AI 2.0's 3.8.6, a `RequireUpperBoundDeps` break the gate caught * Preceded by 1.8.0 (Jun 11), which cleared the long-deferred ACP bump to 0.14.0 * [Version Table](/projects/agentworks-bom) | [Source](https://github.com/markpollack/agentworks) ### Agent Client 0.22.0 **Spring AI 2.0.0 GA + Spring Boot 4.0.7.** The upgrade behind BOM 1.9.0 — primarily a security and dependency release, with one user-facing behavior change. * spring-ai 2.0.0-M2 → 2.0.0 GA and spring-boot 4.0.1 → 4.0.7, clearing the transitive Boot / Spring AI CVEs above * Spring AI 2.0 GA moved to Jackson 3, so `agent-model`, `agent-claude`, `agent-codex`, and `agent-client-core` now declare their Jackson 2 usage explicitly (governed by `jackson-bom 2.21.2`) * Default Gemini model upgraded to `gemini-3.5-flash`; override with `agent-client.gemini.model` * [Project](/projects/agent-client) | [Source](https://github.com/markpollack/agent-client) *** ## Week of June 1, 2026 ### AgentWorks BOM 1.6.0 **The `bom-verification` quality gate lands, with Jackson 2.21.2 convergence.** The largest coordinated release of the week — nearly every member moved — and the first to publish behind an automated gate that proves consumers get a single, converged dependency set. * New `bom-verification/` harness: plain, Spring Boot 3.5, and Spring Boot 4 consumer scenarios with a `JacksonConvergenceTest` and the enforcer `RequireUpperBoundDeps` rule; releases now publish only behind a green gate * Imports `jackson-bom 2.21.2`, converging every member and shielding consumers from stray third-party Jackson transitives * The gate is load-bearing — re-running it against 1.5.0 fails on the exact Jackson divergence 1.6.0 fixes * Bookended by 1.5.0 (Jun 4, a managed test stack — Mockito / JUnit / AssertJ / Byte Buddy) and 1.7.0 (Jun 7, agent-client 0.21.0) * Coordinated bump: claude-code-sdk 1.3.0, agent-client / agent-starter 0.20.0, agent-workflow 0.9.0, agent-judge 0.12.0, agent-memory 0.2.0, agent-journal 1.4.0, agent-hooks 0.6.3, agent-sandbox 0.9.3, agent-bench 0.4.0, experiment-core 0.5.0 * [Version Table](/projects/agentworks-bom) | [Source](https://github.com/markpollack/agentworks) ### Agent Client 0.21.0 **Portable reasoning-effort control across providers.** A new provider-agnostic `effort` option (low / medium / high), with provider-native overrides — and the Provider Parity TCK now actually runs (it had been reporting green without ever executing). * `AgentOptions.getEffort()` — the cross-provider low/medium/high intersection, covered by the TCK * Provider-native overrides win when set: Claude `ClaudeAgentOptions.effort` (→ `--effort`), Codex `CodexAgentOptions.reasoningEffort` up to `xhigh` * 0.20.0 (Jun 6) shipped first: Claude trace content modes — `traceContentMode(FULL | TRUNCATED | LENGTHS)` — and automatic session-transcript archival, powered by agent-journal's TraceWriter v2 * [Portable Options](/docs/agent-client/reference/portable-options) | [Source](https://github.com/markpollack/agent-client) ### Claude Agent SDK 1.3.0 **Aligned to Claude CLI 2.1.162, with a lossless `rawJson` escape hatch.** A compatibility-and-hardening release that pins the wire contract and exposes wire fields the typed API doesn't model yet. * `ParsedMessage.RegularMessage` gains a `rawJson` component carrying the exact stdout line each message was parsed from — backward compatible * All six `HookInput` subtypes now ignore unknown properties, fixing a hook hang under CLI 2.1.162's added PostToolUse fields * New `WireFixtureTest` pins parsing against real 2.1.162 stream-json fixtures * [Project](/projects/claude-agent-sdk) | [Source](https://github.com/markpollack/claude-agent-sdk-java) ### Agent Workflow 0.9.0 **A tool-agnostic `Question` / `Option` callback record.** The `AgentCallback.onQuestion` SPI no longer leaks a tool-library type, and the judge integration completes its move to the `io.github.markpollack` namespace. * New `workflow.callback.Question` record (with nested `Option`) owned by workflow-api, replacing the leaked `AskUserQuestionTool.Question`; the `spring-ai-agent-utils` dependency drops out of workflow-api * Judge integration migrated to `io.github.markpollack.judge` (agent-judge 0.12.0) across `JudgeGate`, `TieredGate`, and the jury loops * [Agent Loop](/docs/agent-workflow/agent-loop) | [Source](https://github.com/markpollack/agent-workflow) *** ## Week of May 26, 2026 ### Agent Client 0.19.0 **JSONL trace file support.** `ClaudeAgentModel` now writes durable trace files during agent execution — tool calls, thinking blocks, text, and result metrics, flushed per event. Set `traceDir` on the builder or `agent-client.claude.trace-dir` in Spring Boot config. Each invocation writes one uniquely-named JSONL file, and the path is exposed via `providerFields.get("tracePath")`. * `TraceTarget` record pairs run ID with trace file path * Fail-fast on invalid trace directory (before client setup) * Full UUID in filenames for collision safety * [Claude Reference](/docs/agent-client/reference/claude-reference#trace-files) | [Source](https://github.com/markpollack/agent-client) ### Agent Workflow 0.8.0 **Trace file capture through the workflow journal.** `AgentClientStep` propagates trace paths from the agent model into `StepTransition` and journal events. Every tool call in a multi-step workflow is now traceable back to a specific JSONL file. * `ExecutionResult(text, tracePath)` — new return type from `AgentClient.executeForResult()` * `AgentContext.TRACE_PATH` — well-known context key, cleared per step * `JdbcTraceRecorder` — `trace_path` column in `step_transitions` table * [What's New](/docs/agent-workflow/whats-new) | [Trace Capture Guide](/docs/agent-workflow/trace-capture) | [Source](https://github.com/markpollack/agent-workflow) ### ACP Java SDK 0.12.0 **7 new protocol methods.** Session lifecycle (list, close, resume) is now stable. Elicitation, fork, and config options land as unstable with the new `@UnstableAcpApi` marker. * [Details](/projects/acp-java-sdk) | [Source](https://github.com/markpollack/acp-java-sdk) ### AgentWorks BOM 1.2.0 Coordinated version bump: agent-client 0.19.0, agent-workflow 0.8.0, agent-journal 1.2.0, ACP Java SDK 0.12.0, and all other managed artifacts. * [Version Table](/projects/agentworks-bom) | [Source](https://github.com/markpollack/agentworks) *** ## Week of May 19, 2026 ### Agent Workflow 0.7.0 **Managed Agents as a step runtime.** `ManagedAgentStep` delegates workflow steps to Anthropic's Managed Agents API. The workflow graph stays in charge — specific steps run in Anthropic's cloud sandbox. * [What's New](/docs/agent-workflow/whats-new#070) | [Source](https://github.com/markpollack/agent-workflow) ### Agent Client 0.18.0 **Migration to markpollack org.** Package rename from `org.springaicommunity.agents` to `io.github.markpollack`, BSL license, standalone POM. * [Migration Guide](/docs/migration/spring-ai-community-to-markpollack) | [Source](https://github.com/markpollack/agent-client)