SkillsBench research · view source on GitHub ↗
runnable-subset-43 — heatmap
Seed baseline, 43 runnable tasks, val split. Best candidate found: seed.
Scope limit — no full candidate × task matrix
iter1.log–iter4.log are 14 lines each and contain no per-task
granular data, so the per-candidate × per-task numbers were never captured for this run.
This table shows the seed baseline per task, annotated with
cand_0002's (the accepted best) pass/partial outcome from
summary.md's own breakdown — not a full per-iteration trace.
| task | seed reward | stderr | trials | cand_0002 outcome |
|---|---|---|---|---|
3d-scan-calc | 0.667 | ±0.333 | 3 | pass (3/3) |
adaptive-cruise-control | 1.000 | ±0.000 | 3 | partial (0.667 (2/3)) |
citation-check | 0.000 | ±0.000 | 3 | — |
court-form-filling | 0.000 | ±0.000 | 3 | partial (0.667 (2/3)) |
dapt-intrusion-detection | 0.000 | ±0.000 | 3 | — |
data-to-d3 | 0.000 | ±0.000 | 3 | — |
drone-planning-control | 0.000 | ±0.000 | 3 | — |
dynamic-object-aware-egomotion | 0.000 | ±0.000 | 3 | — |
edit-pdf | 0.000 | ±0.000 | 3 | pass (3/3) |
energy-ac-optimal-power-flow | 0.000 | ±0.000 | 3 | — |
energy-market-pricing | 0.000 | ±0.000 | 3 | — |
enterprise-information-search | 0.000 | ±0.000 | 3 | — |
exceltable-in-ppt | 0.000 | ±0.000 | 3 | — |
financial-modeling-qa | 0.000 | ±0.000 | 3 | — |
flood-risk-analysis | 0.000 | ±0.000 | 3 | partial (0.333 (1/3)) |
grid-dispatch-operator | 0.000 | ±0.000 | 3 | — |
hvac-control | 1.000 | ±0.000 | 3 | partial (0.333 (1/3)) |
invoice-fraud-detection | 0.333 | ±0.333 | 3 | partial (0.333 (1/3)) |
jax-computing-basics | 0.000 | ±0.000 | 3 | — |
jpg-ocr-stat | 0.000 | ±0.000 | 3 | — |
lake-warming-attribution | 0.000 | ±0.000 | 3 | — |
multilingual-video-dubbing | 0.333 | ±0.333 | 3 | partial (0.667 (2/3)) |
offer-letter-generator | 0.000 | ±0.000 | 3 | — |
organize-messy-files | 0.000 | ±0.000 | 3 | — |
paper-anonymizer | 0.000 | ±0.000 | 3 | — |
pddl-airport-planning | 0.000 | ±0.000 | 3 | — |
pdf-excel-diff | 0.000 | ±0.000 | 3 | — |
pptx-reference-formatting | 0.000 | ±0.000 | 3 | — |
protein-expression-analysis | 1.000 | ±0.000 | 3 | pass (3/3) |
python-scala-translation | 0.000 | ±0.000 | 3 | — |
r2r-mpc-control | 1.000 | ±0.000 | 3 | partial (0.333 (1/3)) |
reserves-at-risk-calc | 0.000 | ±0.000 | 3 | — |
sales-pivot-analysis | 0.000 | ±0.000 | 3 | — |
sec-financial-report | 0.000 | ±0.000 | 3 | — |
shock-analysis-demand | 0.000 | ±0.000 | 3 | — |
shock-analysis-supply | 0.000 | ±0.000 | 3 | — |
simpo-code-reproduction | 0.000 | ±0.000 | 3 | — |
spring-boot-jakarta-migration | 0.000 | ±0.000 | 3 | — |
syzkaller-ppdev-syzlang | 0.667 | ±0.333 | 3 | pass (3/3) |
threejs-structure-parser | 0.000 | ±0.000 | 3 | — |
threejs-to-obj | 0.000 | ±0.000 | 3 | — |
weighted-gdp-calc | 0.000 | ±0.000 | 3 | — |
xlsx-recover-data | 0.000 | ±0.000 | 3 | — |