针对手机助手工具调用,提出校准偏好对齐方法提升推荐精准度。
ToolRec: Calibrated Preference Alignment for Query Recommendation in On-Device Assistants

- 构建708个系统工具库与上下文检索机制,确保推荐可执行。
- 双层校准过滤用户行为噪声,提升工具调用点击信号权重。
- 在1.5亿月活用户平台验证,显著提升点击率与总点击量。
大语言模型虽推动了生成式查询推荐发展,但现有对齐方法多聚焦于通用对话场景,难以满足手机端智能助手对快速调用系统工具的需求。直接使用真实点击日志存在严重噪声,因用户活跃度差异及未突出执行型查询。为此,我们提出ToolRec,一种专为手机端查询推荐设计的校准偏好对齐框架。首先构建包含708个系统工具的SysToolKit,并结合上下文感知工具检索机制,确保推荐结果与用户意图匹配。提出双层级校准机制,分别基于用户活跃度(用户级)和工具调用点击信号(系统级)进行信号优化,有效缓解行为噪声。在此基础上,采用样本级加权的Kahneman-Tversky优化(KTO)对模型进行对齐。在拥有超过1.5亿月活跃用户的OPPO Xiaobu平台开展大规模线上A/B测试显示,ToolRec相较强基线显著提升点击率(CTR)与总点击量,同时保持高查询相关性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have significantly advanced generative query recommendation. However, while alignment is crucial for tailoring LLMs to human preferences, existing alignment methods primarily focus on standard chatbot scenarios, falling short in on-device intelligent assistants where users predominantly expect the rapid invocation of system-level tools. Moreover, directly aligning LLMs with real-world click logs introduces severe noise due to varying user activity levels and the failure to emphasize execution-oriented queries. To address these challenges, we propose ToolRec, a calibrated preference alignment framework tailored for on-device query recommendation. To ground query recommendation with executable tools, we first construct SysToolKit, a comprehensive repository of 708 system tools, paired with a context-aware tool retrieval mechanism to ensure that the extracted tools closely match the user's intent. A dual-level calibration mechanism is then proposed to refine raw click data, effectively mitigating user behavioral noise by calibrating signals based on user activity (user-level), while simultaneously up-weighting click signals on tool-invoking queries (system-level). Guided by these refined preference signals, we then align the model using a sample-level weighted Kahneman-Tversky Optimization (KTO). Extensive online A/B tests on our mobile assistant platform OPPO Xiaobu, which has over 150 million monthly active users, demonstrate that ToolRec can significantly improve Click-Through Rate (CTR) and total click volume over strong baselines while maintaining high query relevance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。