让大模型根据用户偏好调用工具，更懂你的需求。
Advancing and Benchmarking Personalized Tool Invocation for LLMs
- 构建个性化工具调用框架PTool，生成带用户偏好的训练数据。
- 提出首个专用评测基准PTBench，涵盖偏好选择与参数补全任务。
- 适配个人习惯的工具调用，对智能助手研发有直接参考价值。
工具调用是扩展大语言模型能力的关键机制，使其能通过调用外部工具解决复杂问题并获取实时知识。然而，现有研究主要关注模型的基本工具调用能力，忽视了个性化约束。本文提出个性化工具调用新范式，定义两个核心任务：工具偏好（Tool Preference）——在功能相似工具中选择符合用户偏好的选项；以及基于用户档案的查询（Profile-dependent Query）——当查询缺少必要参数时，模型需从用户档案中推断。为此，我们设计了数据合成框架PTool，并构建了首个个性化工具调用评测基准PTBench。通过对多个开源模型进行微调，验证了该框架的有效性，并提供了宝贵实践经验。相关代码与数据已公开于https://github.com/hyfshadow/PTBench。
原文摘要 · Abstract (English)
Tool invocation is a crucial mechanism for extending the capabilities of Large Language Models (LLMs) and has recently garnered significant attention. It enables LLMs to solve complex problems through tool calls while accessing up-to-date world knowledge. However, existing work primarily focuses on the fundamental ability of LLMs to invoke tools for problem-solving, without considering personalized constraints in tool invocation. In this work, we introduce the concept of Personalized Tool Invocation and define two key tasks: Tool Preference and Profile-dependent Query. Tool Preference addresses user preferences when selecting among functionally similar tools, while Profile-dependent Query considers cases where a user query lacks certain tool parameters, requiring the model to infer them from the user profile. To tackle these challenges, we propose PTool, a data synthesis framework designed for personalized tool invocation. Additionally, we construct \textbf{PTBench}, the first benchmark for evaluating personalized tool invocation. We then fine-tune various open-source models, demonstrating the effectiveness of our framework and providing valuable insights. Our benchmark is public at https://github.com/hyfshadow/PTBench.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。