用认知树搜索提升搜索代理的安全性，防止隐蔽攻击。
COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search Agents
- 通过认知树探索生成隐蔽攻击路径，发现潜在风险。
- 在多步交互中识别高危中间操作，实现细粒度监督。
- 仅需少量数据即可实现安全与性能的平衡，适合实际部署。
基于大语言模型的搜索代理具备多步推理和工具调用能力，但此类能力可能导致检索引发的安全退化，因为有害意图可能分解为看似无害的子查询，最终导致不安全结果。现有对齐方法难以捕捉稀疏的安全信号，且无法有效监督多步交互中的多样化违规行为。本文提出COMPASS框架，一种基于认知蒙特卡洛树搜索（Cognitive MCTS）的过程对齐方法，旨在在整个代理工作流中实现稳健的安全对齐，同时保持通用实用性。COMPASS融合认知树探索（CTE），高效合成隐蔽攻击轨迹；并引入内省式逐步对齐（ISA），隔离高风险中间动作以实现细粒度过程监督。实验证明，COMPASS在显著降低训练数据需求的同时，实现了良好的安全-效用权衡。
原文摘要 · Abstract (English)
LLM-powered search agents enable multi-step reasoning and tool use. However, these capabilities introduce retrieval-induced safety degradation, as harmful intents may decompose into seemingly innocuous sub-queries that lead to unsafe outcomes. Existing alignment methods struggle to capture sparse safety signals and fail to supervise diverse violations across multi-step interactions. We propose COMPASS, a Cognitive MCTS-Guided Process Alignment framework designed to achieve robust safety alignment throughout the agent workflow while preserving general utility. COMPASS integrates cognitive tree exploration (CTE) to efficiently synthesize stealthy attack trajectories, and introspective step-wise alignment (ISA) to isolate risky intermediate actions for fine-grained process supervision. Empirical results show that COMPASS achieves a favorable safety-utility trade-off while requiring substantially less training data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。