首个评估个性化网页代理的基准，让AI理解用户历史来应对模糊请求。
Persona2Web: Benchmarking Personalized Web Agents for Contextual Reasoning with User History
- 基于用户历史推断隐性偏好，解决查询模糊问题。
- 多架构实验揭示个性化代理的核心挑战。
- 适合研究智能代理与用户行为建模的研究者。
大语言模型推动了网页代理的发展，但现有代理缺乏个性化能力。由于用户很少明确说明所有细节，实用的网页代理必须能通过推断用户偏好和上下文来理解模糊请求。为解决这一挑战，我们提出Persona2Web，首个在真实开放网络上评估个性化网页代理的基准，基于‘澄清转个性化’原则，要求代理根据用户历史而非显式指令解决歧义。Persona2Web包含：（1）揭示长期隐性偏好的用户历史，（2）需代理推断隐性偏好的模糊查询，（3）支持细粒度评估个性化的推理感知评价框架。我们在多种代理架构、主干模型、历史访问方式及不同模糊度的查询上进行广泛实验，揭示个性化网页代理行为中的关键挑战。代码与数据集已公开于https://serin-kimm.github.io/Persona2Web/，以保障可复现性。
原文摘要 · Abstract (English)
Large language models have advanced web agents, yet current agents lack personalization capabilities. Since users rarely specify every detail of their intent, practical web agents must be able to interpret ambiguous queries by inferring user preferences and contexts. To address this challenge, we present Persona2Web, the first benchmark for evaluating personalized web agents on the real open web, built upon the clarify-to-personalize principle, which requires agents to resolve ambiguity based on user history rather than relying on explicit instructions. Persona2Web consists of: (1) user histories that reveal preferences implicitly over long time spans, (2) ambiguous queries that require agents to infer implicit user preferences, and (3) a reasoning-aware evaluation framework that enables fine-grained assessment of personalization. We conduct extensive experiments across various agent architectures, backbone models, history access schemes, and queries with varying ambiguity levels, revealing key challenges in personalized web agent behavior. For reproducibility, our codes and datasets are publicly available at https://serin-kimm.github.io/Persona2Web/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。