The durable agent harness
cruft agent schedule turns a one-shot agent run into a durable, resumable, multi-turn job stored on disk. An agent can suspend on a timer, a tool call, or a human approval and resume later, even after a process restart. Covers the job model, the subcommands, and the cumulative turn budget.
A single cruft agent run executes an agent once: the process starts, the code runs to completion or is killed by a budget, the audit log is written, and it exits. There is nowhere for the agent to sleep, wait on a slow tool, or ask a human and come back later.
cruft agent schedule turns that one-shot audited sandbox into a durable, resumable, multi-turn job. This is the piece that makes Cruft a harness for agents rather than a sandbox: a long-running agent can suspend at a tool call, a timer, or a human approval; its entire state persists to disk; and it resumes, on the next tick or after a full process restart, exactly where it left off, still inside the same fail-closed, budgeted, fully-audited compartment.
This page assumes the cruft agent overview.
The model: jobs, turns, and the store
- A job is a long-lived record on disk under a store directory. It has an id, a bound policy, and a status.
- A turn is one execution of the agent. Each turn is a fresh
cruft agent runchild process; the scheduler is a pure driver with no daemon and no in-memory state. - When a turn's agent code calls a scheduler primitive (
scheduler.sleep,scheduler.callTool,scheduler.waitForInput), the turn suspends: it records what it is waiting on and exits. The job's status becomesawaiting_timer,awaiting_tool, orawaiting_input. - A later tick (or an
inputcommand, or the timer deadline arriving) resumes the job with a new turn, feeding the previous wait's result back to the agent ascontext.scheduler_resume.
Because the only durable state is an on-disk manifest, not anything held in memory, a job survives a process restart with nothing to reconstruct: the scheduler reads the manifest, does one bounded thing, and writes it back atomically. The manifest binds the job to the exact policy hash and agent-source hash it was created against, so a job cannot be resumed against changed code.
The agent-side surface
Inside a scheduled turn, the agent has a scheduler global (present only when the scheduler drives the turn; an ordinary cruft agent run sees typeof scheduler === "undefined"):
| Primitive | Suspends until | Resume payload (context.scheduler_resume) |
|---|---|---|
scheduler.sleep(ms) | the deadline passes (0 ≤ ms ≤ 86_400_000) | { kind: "timer", await } |
scheduler.callTool(name, args, options) | the durable tool is settled | { kind: "tool", await, result } or { …, error } |
scheduler.waitForInput(kind, payload) | an operator delivers input | { kind: "input", await, result } |
A turn typically branches on whether it is a resume:
// agent.js
if (context.scheduler_resume) {
const r = context.scheduler_resume;
emit({ kind: "resumed", got: r.result }); // handle the delivered result
} else {
emit({ kind: "asking" });
scheduler.callTool("echo", { value: context.goal }); // suspends here
}
Suspending writes a scheduler_await record to the turn's audit log and the job manifest, then ends the turn. There is no ambient setTimeout in an agent compartment at all (see Timers and the availability check); real delay goes through scheduler.sleep.
The subcommands
cruft agent schedule start --project <dir>|--policy <agent-policy.json> [--store <dir>] [--job-id=<id>]
cruft agent schedule tick <job-id> --store <dir>
cruft agent schedule input <job-id> --store <dir> --token <token> --json <json>
cruft agent schedule cancel <job-id> --store <dir>
cruft agent schedule replay <job-id> --store <dir>
cruft agent schedule status <job-id> --store <dir>
cruft agent schedule list --store <dir>
start
Creates a durable job. Exactly one of --project <dir> (a directory holding agent-policy.json) or --policy <path> is required. The policy is validated strictly, the policy source and agent source are hashed, and a fresh manifest is written with status created and turn cursor 0. start prints an agent_scheduler_start line.
--store <dir>is optional forstartonly; it defaults to<policy-dir>/.cruft/agent-scheduler.--job-id=<id>is optional; omitted, a deterministicjob-<hash>id is generated. Ids are 1–128 chars of ASCII alphanumerics plus. _ : -, with no path segments.- An existing job at that id fails (
job already exists).
tick
The workhorse: advances the job by one turn. Requires --store explicitly. It launches one agent-run child, reads whatever the turn suspended on, and writes the new status. tick accepts a job in created, awaiting_timer, or awaiting_tool — not awaiting_input (that advances only via input). A timer job ticked before its deadline reports { "ready": false, "deadline_ms", "now_ms" } and does not launch a turn.
input
Delivers a durable input to an awaiting_input job. --store, --token, and --json are all required; the JSON must parse, and the --token must match the token the agent's waitForInput minted (else input token does not match pending wait). This is the fail-closed gate on human-in-the-loop resume: only the holder of the pending token can advance the job, and only with well-formed data.
cancel, replay, status, list
cancelrevokes a non-terminal job (created/awaiting_*); it is idempotent and empties the pending wait so the agent's resume branch never runs.replayrenders the job's lifecycle from the manifest (below).statuspretty-prints the manifest.listenumerates every job in a store, sorted by id.
Lifecycle
A tick takes a job from created (or from one of the awaiting_* states) through a transient running state and settles it into one outcome:
| Outcome | Reached when |
|---|---|
completed | the turn finished with no further wait |
awaiting_timer | the turn called scheduler.sleep |
awaiting_tool | the turn called scheduler.callTool |
awaiting_input | the turn called scheduler.waitForInput |
failed | the child exited non-zero with no wait recorded |
expired | the cumulative turn budget was exceeded |
completed, failed, cancelled, and expired are terminal. The manifest also carries turn_cursor (the count of turns started) and a derived budget_ledger.
The cumulative turn budget
The one scheduler-specific budget is budgets.max_scheduler_turns in the policy. Unlike the per-turn budgets in cruft agent run, it is cumulative across the whole job. It is enforced before a turn's child launches: if the next turn would exceed the ceiling, the job is set to expired, an agent_scheduler_budget_exceeded line is printed, and the over-budget turn's agent code never runs. With no max_scheduler_turns set, there is no ceiling.
The durable tool RPC
scheduler.callTool(name, args, options) does not run the tool inside the turn. It suspends with a tool wait, and the scheduler settles the call on the next tick and injects the result — the durable-RPC contract. Settlement is doubly fail-closed: the tool must be in the policy's tools list (else not allowed by policy), then it must be a durable tool — today only echo, fail, and slow are settled durably:
echoreturns its ownargsverbatim asscheduler_resume.result.failresolves as a durable error envelope (error.disposition === "error", a stable message) — the turn runs to completion and handles the failure as data, so a tool failure never loses the job. This is durable tool-error resume.slowsimulates latency againstoptions.timeout_ms(default 250 ms). Ifargs.delay_msexceeds the timeout, the scheduler delivers a durable timeout envelope —disposition: "timeout", thetimeout_ms/duration_ms, andlate_result_suppressed: true: the guarantee that a late result will never arrive after the timeout fired, so the resumed turn reacts deterministically. Otherwise it delivers{ ok: true, tool: "slow", delay_ms, value }.
A durable call resolves exactly once: after settlement the job is completed and a further tick is rejected.
The general agent-run tool catalog (readFile,osv.query, thegithub.*readers,model.call,process, …) is not yet wired into durable settlement; calling one throughscheduler.callToolfails durable tool "<name>" is not available. Durable RPC today covers the three built-in tools; in-turn tools (viacallToolin an ordinary run) cover the full catalog.
Replaying a scheduled run
cruft agent schedule replay <job-id> --store <dir> renders the job's lifecycle timeline purely from the manifest — it re-executes nothing. The output is explicit about its own scope with a claim_boundary block:
"claim_boundary": {
"source": "scheduler_manifest",
"per_turn_audit": "not_reconstructed_by_this_replay_slice",
"raw_js_continuation": "not_claimed"
}
That scope marker matters: the durable continuation is the manifest plus the resume-context contract, not a serialized VM heap. For the per-turn detail (the agent's own events, tool records, the scheduler_await), each turn writes to the policy's audit log with a run_id of <job-id>/turn-<n>, and you read those with the ordinary cruft agent replay <audit.jsonl>.
Worked examples
A durable sleep that survives restarts:
# agent.js: if (context.scheduler_resume) emit({kind:"awake"});
# else { emit({kind:"sleeping"}); scheduler.sleep(25); }
cruft agent schedule start --policy ./agent-policy.json --store ./store --job-id=nap
cruft agent schedule tick nap --store ./store
→ status awaiting_timer; the "sleeping" event is logged, "awake" is not
cruft agent schedule tick nap --store ./store # before the deadline
→ {"status":"awaiting_timer","ready":false,"deadline_ms":…,"now_ms":…} (no turn)
# …after 25 ms…
cruft agent schedule tick nap --store ./store
→ status completed; the resumed turn saw context.scheduler_resume.kind === "timer"
A human-approval gate:
# agent.js: scheduler.waitForInput("approval", { subject: "deploy" })
cruft agent schedule tick deploy --store ./store → awaiting_input (mints a token)
cruft agent schedule input deploy --store ./store --token wrong --json '{"approved":true}'
→ exit 76, "input token does not match pending wait" (still awaiting_input)
cruft agent schedule input deploy --store ./store --token <real> --json '{"approved":true}'
→ status completed; the resume delivered result.approved === true
A cumulative turn budget (budgets.max_scheduler_turns: 1):
tick → awaiting_timer, turn_cursor 1
tick → exit 70, {"type":"agent_scheduler_budget_exceeded","limit":1,"attempted_turn":2}
status "expired"; turn 2's agent code never ran
Exit codes
| Code | Meaning |
|---|---|
0 | success (including a suspended awaiting_* and a not-ready timer) |
64 | usage / argument error |
65 | data error (bad policy, corrupt or mismatched manifest, unreadable agent) |
70 | turn failed, or turn-budget expiry |
73 | job already exists |
74 | I/O failure writing the manifest or launching the child |
75 | wrong-state operation (not tickable / not awaiting input / not cancellable) |
76 | fail-closed denial (tool not allowed, durable tool unavailable, token mismatch) |
Limitations
- Three durable tools. Only
echo,fail, andslowsettle durably; the full tool catalog is not yet available throughscheduler.callTool. - No daemon, no automatic wakeup. The scheduler is a pure driver; a timer that has passed its deadline is advanced only when you
tickit. There is no background process that fires at the deadline. - No age budget or job GC.
max_scheduler_turnsbounds turns, not wall-clock time; a suspended job persists until you tick, cancel, or delete it. replayis a manifest summary, not a per-turn or continuation replay (it says so in itsclaim_boundary).