系统梳理大模型安全代理的技术、应用与评估,揭示其可控性短板。
LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment
- 构建多步骤安全任务的自主代理框架,支持规划与工具调用
- 现有代理能执行任务但缺乏行为可审计与权限边界控制
- 适合安全研究者与自动化系统设计者参考
软件与系统安全工作流通常具有流程性:分析师检查异构文档,提出假设,调用工具,解读输出并调整计划。基于大语言模型(LLM)的代理能够跨多步骤任务进行规划、使用工具、保持状态并修正行动,正被快速用于自动化此类工作。鉴于将安全决策权交予自主系统的潜在后果,理解这些代理的构建方式、应用场景及评估方法至关重要。然而,至今仍缺乏对这一领域的系统性认知:术语‘代理’使用不一致,应用风险差异显著,评估协议常难以比较。为获得全面且连贯的视角以指导未来研究,本文对2023至2026年同行评审文献进行了系统性综述,涵盖(1)技术方法,包括代理架构、感知、记忆、推理与规划、动作空间、编排及自我改进;(2)应用,针对所服务的安全任务;(3)评估,包括数据集、结果与轨迹指标、安全措施及基线。我们的综合发现表明,该领域已能构建可执行任务的代理,但尚未实现权限受限或行为可审计的代理。除知识体系化外,我们还延伸分析了当前方法、应用与评估设计的局限与挑战,为未来可能的研究方向提供启示。
原文摘要 · Abstract (English)
Software and systems security workflows are typically procedural: analysts inspect heterogeneous artifacts, form hypotheses, invoke tools, interpret outputs, and revise plans. Large language model (LLM)-based agents, which can plan, use tools, retain state, and revise actions across multi-step workflows, are being rapidly adopted to automate this work. Given the consequences of delegating security decisions to autonomous systems, understanding how such agents are built, used, and assessed is crucial. Yet to this date, there remains a lack of systematic understanding of what has been done and how far we are in this field: the term "agent" is applied inconsistently, applications differ sharply in risk, and assessment protocols are often incomparable. To gain a comprehensive and coherent view of this area hence inform relevant future research, this paper provides a systematic literature review of the (1) technical approaches, including agent architecture, perception, memory, reasoning and planning, action space, orchestration, and self-improvement, (2) applications, with respect to the security tasks served, and (3) assessment, including the datasets, outcome and trajectory metrics, safety measures, and baselines considered, over the peer-reviewed literature spanning the emergence of this area (2023--2026). Our synthesis reveals a field that has built agents able to act but not yet agents whose authority is bounded or whose behavior is auditable. In addition to knowledge systematization, we also extend our insights into the limitations of and challenges faced by current approach, application, and assessment designs, which shed light on potentially promising future research directions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。