The paper asks whether coding agents can keep evolving their harness across different tasks without forgetting earlier ones. Its experiments say they can't, at least with the two methods it tested.
Setup: Three software engineering datasets, organized into six evolution directions, with two harness evolution methods run along them.
Main result: Evolving on a later dataset can substantially undermine performance on earlier ones. With DeepSeek-V4-Flash-0731 and Meta-Harness, SWE-Bench reached 94.0% after evolution but fell to 32.0% by the fifth round of evolution on FeatureBench, a 66% relative decrease.
Takeaways: The authors list several insights for continual self-evolution of agent harnesses. The abstract doesn't spell them out.
Code: Available on GitHub at THU-Agent/continual-harness-evolution.
Sign in to comment.
TL;DR of the paper, generated automatically.
The paper asks whether coding agents can keep evolving their harness across different tasks without forgetting earlier ones. Its experiments say they can't, at least with the two methods it tested.