Agent orchestration

cap-evolve runs in one of two orchestration modes, chosen by a single key in capevolve.yaml. Both share the same honesty guarantees, because those live in the core (seeded split, val-only significance gate, once-only sealed test) and cannot be bypassed by any skill, prompt, or flag in either mode.

orchestration_mode: deterministic   # default — cap-evolve sequences the loop in core
orchestration_mode: agent           # the conversational agent drives the loop itself

Two modes, same guarantees

Deterministic (default)

cap-evolve run sequences the phases itself (intake → implement-and-check → baseline → algorithm → finalize → report). A per-iteration optimizer subprocess (e.g. claude-code, codex, or mock) proposes each edit; the engine evaluates on val, applies the gate, and accepts or rejects. Right when the schedule is known and you want a hands-off, reproducible run. Algorithms: hill-climb, gepa, skillopt.

Agent

cap-evolve run does check → baseline, prints a handoff (run_dir, algorithm, stop_condition) and returns — no algorithm subprocess, no auto-finalize. The same conversational agent that ran intake drives the optimization itself, following the algorithm's Agent-mode loop. It is not a new spawned agent and does not delegate edits — the agent is the optimizer, and you stay in the loop to steer or halt at any round. A Stop hook re-nudges it across turns until the run is finalized.

The `agent-optimize` algorithm — free-form, fully agentic

Every algorithm has an Agent-mode loop, so hill-climb / gepa / skillopt can all be agent-driven. Two algorithms are built for this mode and run only here: agent-optimize (free-form, below) and evograph (collaborative weakness-graph search). agent-optimize has no fixed schedule and no delegated optimizer — the agent owns the whole search, bounded only by the honesty invariants (code-enforced) and the free-text stop_condition it re-reads with the run-dir spend.

Phase 0 — understand first. Read the project, adapter, seed capability, and score()/feedback; note val/test sizes; restate the stop_condition as concrete checks. Ask the user any blocking questions here so the loop then runs unattended.
Free loop. Read the failing-task feedback (free signal); propose one coherent, general edit itself; optionally triage on a cheap task subset; then evaluate on full val and accept only on the paired significance gate (Δ > k·SE) with no regression; snapshot + set-best + log the round.
See constraints every few rounds. Re-read stop_condition and the run-dir spend — score goal, eval cost, optimization cost, and time all fit in one line.
Stop & seal once. When the stop condition is met, seal the held-out test split exactly once and write the report.

When to choose it: when the best next move is judgment — which failure cluster is worth a targeted edit, when a subset eval is enough to kill a bad idea, when the score goal is already met — rather than a fixed round schedule. Prefer a deterministic algorithm when rollouts are cheap and you want a fully hands-off, reproducible run.

Honesty invariants (both modes, code-enforced)

Acceptance and the score-goal check are always on full val through the gate; cheap subset triage never gates. The test split is sealed until a single finalize — the evaluate phase physically restricts --split to train|val; a second finalize raises TestSealError. Every edit encodes a general rule, never a task-specific answer. Every accept is snapshotted via set_best + log_event; every run ends with finalize + report.

How to select it

orchestration_mode: agent
algorithm_skill:    agent-optimize
stop_condition:     "stop when the FULL val mean reward >= 0.78; keep eval + optimization cost modest; you decide which tasks to evaluate and when; no fixed round count"

τ²-bench airline — agent-mode run

The first run driven entirely in agent mode with agent-optimize: the agent understood the benchmark, ran the baseline, then proposed the airline-policy edits itself (payment-construction discipline via the calculate tool was the load-bearing change), gated every candidate on the full val split, and sealed the test once. Runner + user simulator: gpt-oss-120b via a litellm proxy. Split: 30 train == 30 val (fit) · 20 held-out test.

Honest reading — read the caveat, not the headline

Single-trial: val 0.500 → 0.633 (gate-significant), sealed test 0.400 → 0.550. But τ²-bench airline is high-variance: at num_trials=3 the val gain 0.544 → 0.644 is paired-significant while the held-out test 0.467 → 0.400 is not significant, and the 0.78 val target was not reached. Because train == val, the val figure is a fit, not generalization. Full table and caveats: Results.