为大模型工具检索引入风险感知机制,提升安全与效果的平衡。
Risk-Aware Reranking for Agentic Tool Retrieval

- 分离查询相关性与工具风险,通过可调参数控制安全与效用权衡。
- 在Top-k结果中降低高风险工具暴露,实验显示优于纯相关性检索。
- 支持运行时安全评估,适合对安全性要求高的智能体应用。
工具检索决定大模型智能体在用户请求或任务中可访问的外部工具,是执行前的关键安全边界。与文档检索不同,工具检索涉及可执行动作:某个工具对一项任务有用,对另一项可能不必要甚至有风险。现有方法主要优化语义相关性,而安全评估多集中在工具执行后的失败,而非检索阶段引入的风险。本文研究风险感知工具检索,目标是在获取有用工具的同时减少高风险工具的暴露。提出一种轻量级重排序框架,基于冻结的一阶段检索器,分别建模查询相关的相关性和工具级别的暴露风险,通过显式参数调节安全与效用的权衡,利用ToolGraph平滑得分,并可选地施加基于规则的安全约束。为支持检索时的安全评估,我们在UltraTool和Seal-Tools上标注了6,108个工具的五级有序风险等级,并定义了衡量前k个结果中高风险工具暴露程度的指标。在UltraTool和Seal-Tools上的实验表明,该方法在相关性与安全性权衡上优于仅关注相关性的检索器及基线重排序方法,规则过滤版本提供了安全关键场景下的保守操作点。结果表明,检索阶段的过滤能有效缩小智能体执行前的候选动作空间,补充下游工具使用防护措施。代码与附加材料见:https://github.com/qli447/risk-aware-tool-retrieval-release。
原文摘要 · Abstract (English)
Tool retrieval determines which external tools are exposed to an LLM agent for a user query or task, making retrieval a critical pre-execution safety boundary. Unlike document retrieval, tool retrieval exposes executable actions: a tool that is useful for one task may be unnecessary or risky for another. However, existing tool-retrieval methods primarily optimize semantic relevance, and safety evaluations often focus on failures after tool execution rather than risks introduced during retrieval. We study risk-aware tool retrieval, where the goal is to retrieve useful tools while reducing exposure to higher-risk tools. We propose a lightweight reranking framework on top of a frozen first-stage retriever. The framework models query-conditioned relevance and tool-level exposure risk separately, combines them through an explicit parameter controlling the tradeoff between safety and utility, smooths scores over a ToolGraph, and optionally applies rule-based safety constraints. To support retrieval-time safety evaluation, we annotate 6,108 tools across UltraTool and Seal-Tools with five ordinal risk levels and define metrics that measure risky-tool exposure in the top-$k$ results. Experiments on UltraTool and Seal-Tools show that our approach improves the relevance--safety tradeoff over relevance-only retrievers and reranking baselines, with the rule-filtered variant providing a conservative operating point for safety-critical deployments. These findings indicate that retrieval-stage filtering can reduce the candidate action space exposed to agents before execution, complementing downstream tool-use safeguards. The code and supplementary materials are available at: https://github.com/qli447/risk-aware-tool-retrieval-release.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。