HALO is a human-annotated benchmark for proactive agents, the kind that offer help without being asked, and its main finding is that they tend to help too eagerly.
Scope: It covers 4 need types (Dependency, Blockage, Risk, Opportunity) across 10 everyday domains and 50 personal environments. Agents must retrieve evidence before deciding whether to help and how far to go.
Dataset: HALO-8K holds 8,000 responses from 10 models across 400 scenarios, annotated by 20 experts.
Results: The best proprietary and open-source models reach 72.88% and 56.63% acceptability.
Failure mode: Over-proactivity dominates, with premature timing or excessive intensity behind 61.20% of rejected responses.
Sign in to comment.
TL;DR of the paper, generated automatically.
HALO is a human-annotated benchmark for proactive agents, the kind that offer help without being asked, and its main finding is that they tend to help too eagerly.