让AI有自主性需用系统视角，否则会低估其能力与风险
Agentic AI Needs a Systems Theory
- 从系统理论出发，关注智能体间互动带来的新能力
- 简单智能体在环境交互中可自发产生复杂认知行为
- 适合关注AI安全与长期发展的研究者阅读
赋予AI推理能力和一定自主性，被认为是提升其通用性的重要路径。但我们认为，当前对自主智能体的发展亟需更全面的系统理论视角，才能真正理解其能力并防范潜在风险。当前研发过度聚焦单一模型性能，忽视了整体系统中涌现的行为，导致对自主智能体的真实能力与风险严重低估。本文基于大量跨学科文献，阐述了智能体在与环境及其他智能体交互过程中，如何通过基础机制自发产生高级认知、因果推理及元认知意识等能力。最后提出若干关键开放问题与发展建议，强调必须从系统层面理解和塑造自主智能体。
原文摘要 · Abstract (English)
The endowment of AI with reasoning capabilities and some degree of agency is widely viewed as a path toward more capable and generalizable systems. Our position is that the current development of agentic AI requires a more holistic, systems-theoretic perspective in order to fully understand their capabilities and mitigate any emergent risks. The primary motivation for our position is that AI development is currently overly focused on individual model capabilities, often ignoring broader emergent behavior, leading to a significant underestimation in the true capabilities and associated risks of agentic AI. We describe some fundamental mechanisms by which advanced capabilities can emerge from (comparably simpler) agents simply due to their interaction with the environment and other agents. Informed by an extensive amount of existing literature from various fields, we outline mechanisms for enhanced agent cognition, emergent causal reasoning ability, and metacognitive awareness. We conclude by presenting some key open challenges and guidance for the development of agentic AI. We emphasize that a systems-level perspective is essential for better understanding, and purposefully shaping, agentic AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。