实证研究开发者如何监督智能软件代理，揭示四类实际工作方式。
Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agents
- 通过17位开发者的访谈，发现四种监督方式：事前控制、协同规划、实时监控、事后审查。
- 监督不仅是事后纠错，更包含预防性与主动性操作，突破了传统认知。
- 提出实用策略如用测试结果验证代码正确性，适合人机协作设计者参考。
自主软件代理有望提升开发效率，但会出错并表现出新型故障模式，因此人类监督在人机协作中至关重要。现有研究多为概念性框架，缺乏对用户实际监督行为的了解。本文通过访谈17位经验丰富的开发者，开展探索性研究，分析开发者在何时、如何执行监督工作，以及面临的挑战和应对策略。研究发现至少存在四种涌现的监督形式：事前控制、协同规划、实时监控和事后审查。结果表明，监督不仅限于反应性和回顾性，也包含预防性和前瞻性行为。研究还揭示了具体挑战（如难以审查代理生成的代码），并总结了开发者采用的启发式方法（如以测试结果作为代码正确性的保证）。最后，提出高阶启示、未来研究方向、对人本设计与软件工程实践的意义，以及本研究的局限性。
原文摘要 · Abstract (English)
Autonomous software agents hold promise to increase developer productivity but make mistakes and exhibit novel failure modes, making human oversight central to successful human-agent collaboration. Existing research on agent oversight is largely conceptual; normative frameworks exist, but how users actually oversee agents is less known. In this paper, we bridge this gap by providing early empirical anchors for the theoretical discourse on agent oversight. Drawing on interviews with 17 experienced developers, we conduct an exploratory inquiry examining what forms of emergent oversight work developers perform, when, and how. We also document the oversight challenges developers face and the strategies they have started using to address them. We found at least four forms of emergent oversight work: a priori control, co-planning, real-time monitoring, and post hoc review. We show that oversight work is not only reactive and retrospective, as portrayed in existing research, but also preventative and proactive. We describe situated oversight challenges (e.g., difficulty reviewing agent-generated code) and outline heuristics developers adopt to address such challenges (e.g., using test results as guarantees for code correctness). We conclude with high-level takeaways, future research directions, implications for the human-centered design of software agents and for software engineering practice, and limitations of our research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。