Antbench
JANUS asked whether prefetch is measurable in synthetic workflows; Antbench asks whether it survives contact with real software. It instruments 57 supervisor–worker delegations across 9 repositories from SWE-bench Verified, with ground truth derived mechanically from diffs and test runs rather than from a judge. Workers turn out to waste 34.6% of their retrievals, and free IDF matching recovers half of what a worker will actually need (recall@8 of 0.50 against 0.06 for random, p < 0.0001).
- SWE-bench
- Agent Delegation
- Retrieval