Agent Configuration

Agent YAML Format

This page describes the agent process contract in the 0.6.0 release.

command: <shell command to run>
timeout: <ISO 8601 duration>

Agent Bench passes the command to bash -c with the task workspace as the working directory. The command should read INSTRUCTION.md, change the workspace, and exit. It runs as a local host process with the invoking user’s permissions.

The workspace directory is an organizational boundary, not a security boundary. Agent Bench does not provide container execution or isolation. Use your own disposable VM, CI runner, or other externally managed isolation for untrusted agents or benchmark definitions. Benchmark setup and post commands have the same host authority as the configured agent command.

Examples

# Claude Code
command: claude --print --dangerously-skip-permissions "Read INSTRUCTION.md and follow the instructions precisely."
timeout: PT45M

# Gemini CLI
command: gemini -p "Read INSTRUCTION.md and follow the instructions."
timeout: PT30M

# Local script
command: ./my-agent.sh
timeout: PT10M

#!/bin/bash
instruction=$(cat INSTRUCTION.md)
# Use the instruction in your agent logic.
printf '%s\n' 'Hello World!' > hello.txt

Filesystem Contract

After provide, the workspace contains:

File	Description
`INSTRUCTION.md`	Task description
`.bench-context.yaml`	Benchmark, task, version, and timeout context
Source files	Workspace-template files when supplied by the task

The external agent may create or modify files in this directory. Agent Judge implementations inspect that result during grading.

Optional Agent Journal

If the agent writes journal.yaml, Agent Bench records efficiency metrics with the trial:

schema: bench.journal.v1
totalTurns: 8
totalInputTokens: 4000
totalOutputTokens: 2000
totalCostUsd: 0.12
durationMs: 15000

Agents without a journal are still graded; only the journal-derived efficiency fields are absent.

Optional Trajectory Reference

If the agent writes trajectory-ref.txt containing a path or URI, Agent Bench copies the reference into the trial result for later analysis. Agent Bench does not upload or manage the referenced trace.

Projects

AgentWorks

Supporting Projects

Migration

Agent YAML Format

Examples

Filesystem Contract

Optional Agent Journal

Optional Trajectory Reference

​Agent YAML Format

​Examples

​Filesystem Contract

​Optional Agent Journal

​Optional Trajectory Reference

Agent YAML Format

Examples

Filesystem Contract

Optional Agent Journal

Optional Trajectory Reference