用真实职场任务评估AI代理的生产力，更贴近商业价值。
xbench: Tracking Agents Productivity Scaling with Profession-Aligned Real-World Evaluations
- 基于行业专家定义的真实任务，评估AI在招聘与营销中的表现。
- 招聘任务50个，营销任务覆盖50个广告需求与836名网红。
- 可追踪技术与市场匹配度，适合关注AI落地的企业和研究者。
我们提出xbench，一个动态、职业对齐的评估套件，旨在弥合AI代理能力与真实世界生产力之间的差距。现有基准多聚焦孤立技术技能，难以反映代理在专业场景中的经济价值。xbench针对具有商业意义的领域，由行业专业人士定义评估任务，构建与生产力价值强相关的指标，支持预测技术-市场契合度（TMF），并可长期追踪产品能力。初始实现包含两个基准：招聘与营销。招聘基准从真实猎头业务场景中收集50个任务，评估代理在公司画像、信息检索与人才搜寻方面的能力；营销基准则通过836名候选网红池，评估代理在匹配广告主50项需求上的表现。我们对主流现代代理进行了初步评估，建立了这些专业领域的基准线。持续更新的评估集与评测结果可在https://xbench.org获取。
原文摘要 · Abstract (English)
We introduce xbench, a dynamic, profession-aligned evaluation suite designed to bridge the gap between AI agent capabilities and real-world productivity. While existing benchmarks often focus on isolated technical skills, they may not accurately reflect the economic value agents deliver in professional settings. To address this, xbench targets commercially significant domains with evaluation tasks defined by industry professionals. Our framework creates metrics that strongly correlate with productivity value, enables prediction of Technology-Market Fit (TMF), and facilitates tracking of product capabilities over time. As our initial implementations, we present two benchmarks: Recruitment and Marketing. For Recruitment, we collect 50 tasks from real-world headhunting business scenarios to evaluate agents' abilities in company mapping, information retrieval, and talent sourcing. For Marketing, we assess agents' ability to match influencers with advertiser needs, evaluating their performance across 50 advertiser requirements using a curated pool of 836 candidate influencers. We present initial evaluation results for leading contemporary agents, establishing a baseline for these professional domains. Our continuously updated evalsets and evaluations are available at https://xbench.org.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。