神经代理通过分解与组合，零样本学会描述新图像。
CtD: Composition through Decomposition in Emergent Communication
- 先分解图像为基础概念，再用代码本组合成描述
- 在未见过的图像上实现零样本泛化，无需额外训练
- 适合研究语言涌现、多智能体协作与可解释生成
组合性是一种认知机制，使人类能够系统性地以新颖方式组合已有概念。本研究展示了人工神经代理如何习得并运用组合泛化能力来描述此前未见的图像。提出的方法称为「通过分解进行组合」（Composition through Decomposition），包含两个顺序训练阶段：在『分解』阶段，代理通过多目标协同游戏中的交互学习从图像中提取基本概念，并利用获得的代码本；在『组合』阶段，代理使用该代码本将基本概念组合成复杂语句以描述新图像。值得注意的是，我们在『组合』阶段观察到零样本泛化现象，即无需额外训练即可对未见过的图像进行有效描述。
原文摘要 · Abstract (English)
Compositionality is a cognitive mechanism that allows humans to systematically combine known concepts in novel ways. This study demonstrates how artificial neural agents acquire and utilize compositional generalization to describe previously unseen images. Our method, termed "Composition through Decomposition", involves two sequential training steps. In the 'Decompose' step, the agents learn to decompose an image into basic concepts using a codebook acquired during interaction in a multi-target coordination game. Subsequently, in the 'Compose' step, the agents employ this codebook to describe novel images by composing basic concepts into complex phrases. Remarkably, we observe cases where generalization in the `Compose' step is achieved zero-shot, without the need for additional training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。