SkillsBench research · view source on GitHub ↗
transfer-eval-8fold — heatmap
Zero-shot skill transfer, 8 folds, generated 2026-09-02.
Read this before the table
- Every fold reports best_id='seed', test_reward == test_baseline_reward, test_delta == 0.0. This is an artifact of --max-iterations 0: the transfer project's seed IS the frozen donor skill and is the only artifact evaluated, so cap-evolve compares it against itself. Real transfer effect = transfer_reward - native_seed.
- For tasks with status KILLED_ceiling, native_optimized is the in-loop val score (results.json 'best'), not a held-out test score, because final_test is null - the run was stopped on saturation before a held-out eval ran.
- 8 folds over 5 tasks in 2 domains; directional only.
| # | train task | test task | job | status | transfer reward | native seed | native optimized | transfer − native seed |
|---|---|---|---|---|---|---|---|---|
| 1 | shock-analysis-demand | shock-analysis-supply | 555427 | DONE | 0.10 | 0.00 | 0.20 | +0.10 |
| 2 | shock-analysis-demand | weighted-gdp-calc | 540431 | DONE | 0.20 | 0.80 | 1.00 ‡ | -0.60 |
| 3 | shock-analysis-supply | shock-analysis-demand | 543167 | DONE | 0.00 | 0.00 | 0.90 | 0.00 |
| 4 | shock-analysis-supply | weighted-gdp-calc | 540433 | DONE | 0.30 | 0.80 | 1.00 ‡ | -0.50 |
| 5 | weighted-gdp-calc | shock-analysis-demand | 543168 | DONE | 0.00 | 0.00 | 0.90 | 0.00 |
| 6 | weighted-gdp-calc | shock-analysis-supply | 543169 | DONE | 0.10 | 0.00 | 0.20 | +0.10 |
| 7 | exam-block-sequencing | paratransit-routing | 543170 | DONE | 0.20 | 0.00 | 1.00 ‡ | +0.20 |
| 8 | paratransit-routing | exam-block-sequencing | 543171 | DONE | 0.40 | 0.10 | 1.00 ‡ | +0.30 |
‡ in-loop val score, not a held-out test score (task is KILLED_ceiling in results.json).