开发者在构建智能体系统时，面临能力与风险难以兼顾的困境。
The Perils of Agency: How Developers Perceive, Prioritize, and Address Risks in Agentic AI Products
- 开发者将自主性、工具使用等特性视为风险源头
- 优先处理产品和商业风险，忽视社会影响如隐私与就业
- 缺乏有效控制手段，常被迫削弱智能体核心功能
智能体AI系统具备自主决策、调用工具、适应环境等能力，但这些特性也可能引发或加剧产品风险。我们对35名产业开发者进行了研究，发现其对风险的认知与其智能体特性紧密相关，如自主性、工具使用及真实场景应用。开发者优先关注产品和业务风险，而非下游社会风险（如失业与用户隐私）。这种优先级影响了他们缓解风险的能力与意愿。此外，开发者缺乏成熟的风控机制，往往通过限制使智能体有用的特性（如自主性和目标复杂度）来降低风险。研究揭示出智能体开发中能力与风险控制之间的矛盾：需应对由智能体能力带来的风险，却缺乏不牺牲功能的支持手段。
原文摘要 · Abstract (English)
Agentic AI systems act autonomously, use tools, adapt to context, and operate in complex real-world environments. However, these same characteristics can create or exacerbate product risks. We studied how industry developers (n=35) perceive, prioritize, and address the risks in their agentic AI products. We found that developers' perceptions of risk were closely tied to the qualities that made the product agentic, such as autonomy, tool use, and usage in a real-world context. Developers prioritized product and business risks before considering downstream societal risks like job displacement and end-user privacy. This prioritization also impacted developers' ability and motivation to mitigate agentic risks. Finally, developers lacked mature controls for containing agentic risks, often relying on constraining the same characteristics that make agents useful: e.g., autonomy and goal complexity. These findings reveal a capability vs. risk control tension in agentic AI development: developers need to address risks that emerge from agentic capabilities, yet they currently have limited support for doing so without constraining agentic functionality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。