arXiv:2503.04940cs.CLcs.AI2025-03

用向量量化让智能体先内部练语言,再对外沟通更有效。

VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization

  • 用向量量化构建可微的离散符号生成机制
  • 自玩预训练后符号对齐率提升,任务成功率更高
  • 适合研究语言涌现、分布式强化学习的学者

涌现语言(EL)关注人工智能体间通信的自发形成。尽管符号化通信更贴近人类语言的离散特性,但因符号采样不可导,学习此类协议仍具根本挑战。现有方法依赖高方差梯度估计(如REINFORCE)或连续松弛(如Gumbel-Softmax),在训练稳定性和可扩展性上存在局限。受认知理论启发,我们探索在相互交互前通过自玩(self-play)促进语言涌现。提出向量量化涌现语言(VQEL),将向量量化引入消息生成过程,使智能体能基于学习到的码本进行离散内部表征的自玩,同时保持端到端可微。所得向量量化码本自然生成可直接转移和对齐的符号词汇表。实验证明,经VQEL自玩预训练的智能体,在后续相互交互中表现出更强的符号一致性与更高的任务成功率。这些结果表明,自玩是学习离散通信协议的有效且原则性机制,解决了涌现语言系统中的优化与表征难题。

原文摘要 · Abstract (English)

Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more closely mirror the discrete nature of human language, learning such protocols remains fundamentally difficult due to the non-differentiability of symbol sampling. Existing approaches typically rely on high-variance gradient estimators such as REINFORCE or on continuous relaxations such as Gumbel-Softmax, both of which suffer from limitations in training stability and scalability. Motivated by cognitive theories that emphasize intrapersonal processes preceding communication, we explore self-play as a substrate for language emergence prior to mutual interaction. We introduce Vector Quantized Emergent Language (VQEL), a novel architecture that incorporates vector quantization into the message generation process. VQEL enables agents to perform self-play using discrete internal representations derived from a learned codebook while preserving end-to-end differentiability. Moreover, the resulting vector-quantized codebook naturally induces a symbolic vocabulary that can be directly transferred and aligned during subsequent mutual play with other agents. Empirical results show that agents pretrained via VQEL self-play achieve more consistent symbol alignment and higher task success when later engaged in mutual interaction. These findings position self-play as a principled and effective mechanism for learning discrete communication protocols, addressing key optimization and representational challenges in emergent language systems.

语言涌现自玩向量量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。