A Microsoft paper introduces ParetoSaddler, a way to tune LLM agent harnesses for both quality and cost without changing model weights.
The idea: Intermediate candidates are promoted separately from the final deployment choice, so a candidate that looks costly early isn't discarded before it can help later. Promotion uses Bayesian bootstrap Pareto support across repeated runs.
ClawMark: Quality rises from 70.86% to 81.35%, and cost falls from $1.17 to $1.01 per task.
GAIA2: Quality rises from 56.22% to 62.89%, and cost falls from $0.78 to $0.45 per task.
Code: The submitter links a GitHub repo (microsoft/AutoSaddler, feat/paretosaddler branch).
Sign in to comment.
TL;DR of the paper, generated automatically.
A Microsoft paper introduces ParetoSaddler, a way to tune LLM agent harnesses for both quality and cost without changing model weights.
Better answers, lower bills, on both benchmarks.