Architecture

cap-evolve is a library of Agent Skills over a tiny pure-stdlib core (core/cap_evolve). Six phases run left-to-right; only the algorithm phase loops, and the gate is what makes each accepted iteration a real improvement.

The cap-evolve pipeline Six sequential phases — intake, check, baseline, algorithm, finalize, report — with the algorithm phase expanded to show its per-iteration inner loop: diagnose, propose, evaluate, gate. On rejection, control returns to diagnose. On acceptance, the iteration is committed and the loop advances. intake check (hard gate) baseline algorithm iterate ×N finalize seal test report algorithm — each iteration diagnose cluster failures propose optimizer edit evaluate N trials on val gate Δ > k·SE rejected — try again (protect-set + IMPACT + JOURNAL) accepted → git-commit iteration → memory updated → next iter or exit Repeats until iteration budget is exhausted; best candidate advances to finalize.
The pipeline runs left-to-right; the algorithm phase is the only one that loops, and the gate is what makes each accepted iteration a real improvement.

The six phases

1 · intake
Interviews you, installs the benchmark, and wires the adapter, trajectory path, and optimizer prompt — all before any budget is spent. Missing NEEDED inputs are asked for, never fabricated.
2 · implement-and-check
The HARD GATE. cap-evolve check refuses to proceed until every adapter method is real and score() is deterministic — no spend against a stub.
3 · baseline
Freezes the seeded train/val/test split (written once, test sealed) and scores the unmodified seed on val: the candidate every iteration must beat.
4 · algorithm
hill-climb / gepa / skillopt. The only phase that loops: diagnose → propose → evaluate → gate, git-committed on accept.
5 · finalize
Scores the best candidate on the sealed test split exactly once — the run dir enforces the seal.
6 · report
Writes report.md and a self-contained dashboard.html.

Each algorithm iteration: diagnose failing val traces into failure clusters → the optimizer proposes a large, multi-part edit → the candidate is evaluated on val (each of N trials gets its own seed, so pass^k measures real variance) → a paired significance gate (Δ > k·SE, val-only) accepts or rejects → the iteration is git-committed and memory updated.

See HONEST_EVAL.md for the splitting / gating / sealing guarantees and ADAPTER_CONTRACT.md for the adapter.

What the optimizer receives each iteration

The harness assembles a capability-scoped working dir per iteration, then runs your chosen coding-agent CLI in it:

  • The selected capability skill(s) — both as ./guidance/<cap>/ and placed natively in the agent's own skills dir (e.g. .claude/skills/) so a headless agent auto-loads them. Each carries a "What you can change here" menu and edit boundaries.
  • The diagnose method (./guidance/diagnose/) — how to cluster failures into a reflective dataset (per failing task: Inputs, Generated Outputs, Feedback).
  • Only the current best step's full trajectories (./trajectories/) — the runner's verbatim traces of the candidate it builds on, never the seed + every rejected attempt.
  • Supporting sources / data model (./guidance/sources/) — the capability_sources files, copied verbatim so new tool code is written against the real types.
  • Per-task IMPACT of prior candidates — which task ids each prior edit BROKE (were passing) and FIXED, plus the currently-passing set to protect — causal feedback so a known regression is never re-introduced.

Cross-iteration files (clean ownership)

FileOwnerPurpose
LEDGER.mdframeworkFACTS: every iteration's outcome + the exact tasks it broke/fixed
JOURNAL.mdoptimizerappend-only HANDOVER across the run (tried / worked / regressed / refuted / focus-next)
PROCESS.mdoptimizerEXPLAINABILITY, snapshotted per candidate
RUNMAP.md + prior_iterations/frameworka manifest plus every prior iteration's PROCESS.md and capability diff, for real prior-work access

Because it sees all failure clusters, the protect-set, and the prior causal impact at once, the optimizer produces one bold, multi-part candidate per iteration that addresses every cluster without regressing the wins — not a one-line tweak.

What the optimizer can change

The prompt and the tools are equally fair game:

  • Prompt (system-prompt) — rewrite/consolidate/add rules, add examples, tighten the output contract, but never drop a needed rule (change / consolidate / add, don't delete).
  • Tools (tools) — add/replace/wrap tools, edit tool CODE for deterministic enforcement, improve docs and return values (actionable errors), add loop/workflow/composite tools, and swap via a safe wrapper — never bare-remove a primitive. A tool body the model cannot skip beats a sentence it can forget; a knowledge-gap failure still belongs in the prompt.

Speed + observability

All N trials of a candidate run in one concurrent pass when the adapter implements run_trials(tasks, ctx, *, n_trials, base_seed) (per-trial persistence and pass^k/SE are byte-for-byte unchanged). The live dashboard shows intake cost/time, per-iteration optimizer & runner cost + time, the cumulative-best stair, a tasks × iterations pass/fail heatmap, per-iteration git diffs, the lineage tree, and gate decisions.

Skill library

22 skills · 5 algorithms (3 run-executable + 2 agent-mode) · 14 optimizer backends, all over the core. Extending is one folder or one registry row — see EXTENDING.md.

How the counts are defined. A skill is one skills/<component>/<name>/SKILL.md, i.e. one row of the generated skills/_registry/manifest.json — phases, capabilities, algorithms, optimizers and orchestrate all count. An algorithm is a skill whose meta.yaml says component: algorithm; agent-mode-only algorithms count too. An algorithm is run-executable when cap-evolve run drives its deterministic loop; it is agent-mode when its meta.yaml declares agent-mode only, in which case cap-evolve run hands off to the conversational agent after baseline and the skill's scripts/run.py is a loud guard rather than a loop (orchestration_mode: agent is required). An optimizer backend is one top-level row of skills/optimizers/registry.yaml. python skills/_registry/build_manifest.py skills prints the skill and algorithm numbers and core/tests/test_advertised_counts.py asserts all of them against the repo, so every count above is checkable rather than remembered.

ComponentSkills
orchestrateorchestrate · using-cap-evolve
phasesintake · implement-and-check · baseline · evaluate · diagnose · gate · finalize · report
capabilitiessystem-prompt · skill-package · tools · mcp-tool
algorithmshill-climb (--focus all|cyclic|hardest-first) · gepa · skillopt · agent-optimize (agent mode only) · evograph (agent mode only)
optimizersrun-optimizer + optimizers/registry.yaml (14 backends incl. mock)
Claude Code plugin

claude --plugin-dir ./plugins/cap-evolve exposes every skill as /cap-evolve:<skill> and arms honesty hooks (PreToolUse denies edits to the sealed test/gold; Stop/SubagentStop block finishing until cap-evolve check and the gate are green) — all in core-owned scripts, never in editable skill markdown.