通过行为感知验证，让智能体框架进化更高效省资源。
Verify Smarter, Evolve Further: Efficient Harness Evolution through Behavior-Aware Verification
- 基于执行轨迹生成修改建议，只在相关任务上验证。
- 在三个框架上提升7.6%~13.6%性能，节省大量评估开销。
- 适合资源受限下优化智能体框架的研究者使用。
智能体框架决定语言模型代理如何使用指令、工具和运行时组件，但调整这些框架需高成本验证。现有提出-验证方法通常对每个候选在固定任务集上评分，浪费推演在无关行为上，且聚合得分可能掩盖具体退化。我们提出HarnessLens，一种预算感知的自动化框架演化框架。HarnessLens联合探索任务空间与用户可配置组件，从执行轨迹中提取候选修改，并通过可归因证据门控选择行为相关任务进行针对性验证。在三个智能体框架和四个基准上，HarnessLens将平均保留性能提升7.6%-13.6%，同时显著低于基线的评估预算消耗。结果表明，在有限交互预算下，具备显式归因的行为感知验证能实现更可靠、更高效的框架演化。代码已公开于https://github.com/jhxu5214/HarnessLens。
原文摘要 · Abstract (English)
Agent harnesses shape how language-model agents use instructions, tools, and runtime components, but adapting these harnesses requires costly verification. Existing propose-and-verify methods typically score every candidate on a fixed task set, wasting rollouts on unrelated behaviors and allowing aggregate scores to obscure specific regressions. We introduce HarnessLens, a budget-aware framework for automated harness evolution. HarnessLens jointly explores the task space and user-configurable components, derives candidate modifications from execution trajectories, and selectively verifies each candidate on behavior-relevant tasks using an attributable-evidence gate. Across three agent harnesses and four benchmarks, HarnessLens improves average held-out performance by 7.6-13.6% while consuming substantially less evaluation budget than competing baselines. These results demonstrate that behavior-aware verification with explicit attribution enables more reliable and sample-efficient harness evolution under constrained interaction budgets. Our code is available at https://github.com/jhxu5214/HarnessLens.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。