Adapter templates

Onboarding a benchmark should be config, not code. Instead of writing an adapter from scratch, copy a ready-made template, drop in the shared model_config.py, set credentials in a .env, and run. You only write code when task loading or scoring is genuinely custom.

cp -r templates/adapters/<template>/*  .capevolve/project/adapters/
cp    templates/adapters/model_config.py  .capevolve/project/adapters/
mv    .capevolve/project/adapters/seed_capability  .capevolve/project/seed_capability
# set MODEL + credentials in a repo-root .env, then:
cap-evolve check && cap-evolve run

Which template?

Template	Best for	Optimizes	Task source
`jsonl_litellm/`	The common case — start here	a system prompt	a local `tasks.jsonl`
`huggingface_litellm/`	any HuggingFace eval dataset	a system prompt	`datasets.load_dataset(...)`
`tau2_bench/`	tau2-bench airline	system-prompt policy + tool code	tau2's runner
`skillsbench/`	SkillsBench	shared Agent Skills	BenchFlow `bench eval run`
`swe_bench/`	SWE-bench / Lite	coding-agent prompt	HuggingFace + Docker harness

The first two are generic — point them at your data with env vars, no code edits. The last three are worked benchmark adapters you copy and run.

One line to switch provider

All templates wire the model through model_config.py, which resolves credentials from env vars by the MODEL prefix (the same routing litellm does). It is lazy — no network at import — so cap-evolve check stays offline. Switching providers is a one-line MODEL= change, no adapter edits:

MODEL=gpt-4.1-mini                         OPENAI_API_KEY=sk-…            # OpenAI
MODEL=anthropic/claude-sonnet-4-6          ANTHROPIC_API_KEY=sk-ant-…     # Anthropic
MODEL=vertex_ai/claude-sonnet-4-6                                         # Vertex AI (ADC — no key)
MODEL=azure/gpt-4o   AZURE_API_KEY=…       AZURE_API_BASE=https://….openai.azure.com   # Azure
MODEL=ollama/qwen2.5:7b-instruct           API_BASE=http://localhost:11434  # Ollama (local)
MODEL=litellm_proxy/my-model  LITELLM_PROXY_API_BASE=http://proxy:4000  LITELLM_PROXY_API_KEY=sk-…  # any proxy
MODEL=openai/my-model  OPENAI_API_KEY=…    OPENAI_API_BASE=http://my-endpoint/v1        # any OpenAI-compatible

The common case — `jsonl_litellm`

Tasks are one JSON object per line — {"id", "input", "target"}. Point the adapter at your file, pick a scoring mode, set a model; the optimizer edits the system prompt in seed_capability/prompt.txt.

TASKS_FILE=/path/to/tasks.jsonl     # {"id","input","target"} per line
SCORING=exact                       # exact | contains | regex
MODEL=gpt-4.1-mini
OPENAI_API_KEY=sk-…

Full per-template env-var tables, prerequisites, the provider matrix, and the “write your own” guide are in docs/ADAPTER_TEMPLATES.md; the templates themselves are in templates/adapters/.

Adapter templates

Which template?

One line to switch provider

The common case — jsonl_litellm

The common case — `jsonl_litellm`