模型无关强化学习代理展现类似直觉型意图行为
Model-Free RL Agents Demonstrate System 1-Like Intentionality
- 将人类直觉与规划认知类比为模型无关与模型依赖强化学习
- 发现无需显式规划的代理也能表现出结构化意图行为
- 适用于思考AI责任归属与伦理治理的跨学科研究
本文提出，尽管模型无关强化学习（RL）代理缺乏显式规划机制，其行为仍可类比于人类认知中的系统1（快速思考）过程。与依赖内部表征进行规划的模型依赖代理（类比系统2，慢速思考）不同，模型无关代理仅对环境刺激做出反应，不进行前瞻性建模。本文构建了一个新框架，将系统1与系统2的认知二分法映射到模型无关与模型依赖强化学习之间。该视角挑战了‘意图性必须依赖规划’的主流假设，表明意图性可体现在模型无关代理的结构化、反应式行为中。通过融合认知心理学、法律理论与实验法学的跨学科洞见，本文探讨了这一观点在责任归属与AI安全中的意义，倡导对强化学习系统的意图性采取更广泛、情境化的理解，为伦理化部署与监管提供依据。
原文摘要 · Abstract (English)
This paper argues that model-free reinforcement learning (RL) agents, while lacking explicit planning mechanisms, exhibit behaviours that can be analogised to System 1 ("thinking fast") processes in human cognition. Unlike model-based RL agents, which operate akin to System 2 ("thinking slow") reasoning by leveraging internal representations for planning, model-free agents react to environmental stimuli without anticipatory modelling. We propose a novel framework linking the dichotomy of System 1 and System 2 to the distinction between model-free and model-based RL. This framing challenges the prevailing assumption that intentionality and purposeful behaviour require planning, suggesting instead that intentionality can manifest in the structured, reactive behaviours of model-free agents. By drawing on interdisciplinary insights from cognitive psychology, legal theory, and experimental jurisprudence, we explore the implications of this perspective for attributing responsibility and ensuring AI safety. These insights advocate for a broader, contextually informed interpretation of intentionality in RL systems, with implications for their ethical deployment and regulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。