arXiv:2603.04746cs.AIcs.HC2026-03

提出新框架应对智能体式AI带来的持续对齐难题

Visioning Human-Agentic AI Teaming: Continuity, Tension, and Future Research

  • 用团队态势感知理论重构人机协作中的共同认知机制
  • 发现传统协同逻辑在开放自主系统中面临动态失效风险
  • 为未来人机长期协同研究提供关键方向指引

人工智能正经历结构性变革,以具备开放行动轨迹、生成性输出和动态目标的自主系统为特征。这类系统在人机协作(HAT)中引入结构性不确定性,体现在行为路径、认知基础和治理逻辑的持续演变上。此时,对齐不能依赖静态输出共识,而需随计划推进与优先级变化持续维持。本文拓展团队态势感知(Team SA)理论,基于共享感知、理解与预测,构建该转型的整合锚点。尽管Team SA仍是分析基础,其稳定逻辑假设一旦达成共享认知即可通过迭代更新支持协调行动,但自主系统挑战这一前提。论证分两步:首先重构人与AI在开放自主下的认知状态,包括异构系统间预测一致性的情境理解;其次检验传统关系互动、认知学习、协调控制等动态过程在自适应自治下的有效性。通过区分连续性与张力,厘清既有洞见的适用边界与结构不确定性的压力来源,提出面向未来的HAT研究议程。核心挑战并非即时共识,而是面对持续生成、修订、执行与治理的未来时,能否保持长期对齐。

原文摘要 · Abstract (English)

Artificial intelligence is undergoing a structural transformation marked by the rise of agentic systems capable of open-ended action trajectories, generative representations and outputs, and evolving objectives. These properties introduce structural uncertainty into human-AI teaming (HAT), including uncertainty about behavior trajectories, epistemic grounding, and the stability of governing logics over time. Under such conditions, alignment cannot be secured through agreement on bounded outputs; it must be continuously sustained as plans unfold and priorities shift. We advance Team Situation Awareness (Team SA) theory, grounded in shared perception, comprehension, and projection, as an integrative anchor for this transition. While Team SA remains analytically foundational, its stabilizing logic presumes that shared awareness, once achieved, will support coordinated action through iterative updating. Agentic AI challenges this presumption. Our argument unfolds in two stages: first, we extend Team SA to reconceptualize both human and AI awareness under open-ended agency, including the sensemaking of projection congruence across heterogeneous systems. Second, we interrogate whether the dynamic processes traditionally assumed to stabilize teaming in relational interaction, cognitive learning, and coordination and control continue to function under adaptive autonomy. By distinguishing continuity from tension, we clarify where foundational insights hold and where structural uncertainty introduces strain, and articulate a forward-looking research agenda for HAT. The central challenge of HAT is not whether humans and AI can agree in the moment, but whether they can remain aligned as futures are continuously generated, revised, enacted, and governed over time.

人机协作自主智能对齐问题

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。