发现多智能体大模型共识存在表征坍塌，提出无需训练的多样性加权方案提升准确率。
Representational Collapse in Multi-Agent LLM Committees: Measurement and Diversity-Aware Consensus
- 通过嵌入链式思维并计算相似度，量化智能体间表征冗余
- 新方法在GSM8K上达87%准确率，比自洽性低26%的耗能
- 揭示嵌入模型选择直接影响协作效果，适合多智能体系统设计者
多智能体大模型委员会通过不同角色提示运行相同模型，并以多数投票聚合输出，隐含假设各智能体提供互补证据。我们对每个智能体的思维链进行嵌入，并测量成对相似度：在100个GSM8K问题上，使用三个Qwen2.5-14B智能体，平均余弦相似度为0.888，有效秩仅为2.17（满分3.0），这一失败模式称为表征坍塌。提出的无训练共识协议DALC，基于嵌入几何计算多样性权重，在GSM8K上达到87%准确率，较自洽性方法提升3个百分点，且消耗降低26%。消融实验表明，单次运行方差为1-3分，提示共享比单纯多样性加权贡献更大，且编码器选择显著影响坍塌程度（mxbai余弦0.908，nomic为0.888）和下游性能。更稳健的结论是：坍塌可测量，随任务难度加剧，嵌入代理的选择是任何潜在通信协议的首要设计决策。
原文摘要 · Abstract (English)
Multi-agent LLM committees replicate the same model under different role prompts and aggregate outputs by majority vote, implicitly assuming that agents contribute complementary evidence. We embed each agent's chain-of-thought rationale and measure pairwise similarity: across 100 GSM8K questions with three Qwen2.5-14B agents, mean cosine similarity is 0.888 and effective rank is 2.17 out of 3.0, a failure mode we term representational collapse. DALC, a training-free consensus protocol that computes diversity weights from embedding geometry, reaches 87% on GSM8K versus 84% for self-consistency at 26% lower token cost. Ablation experiments reveal 1-3 point per-protocol run-to-run variance, confirm that hint sharing contributes more than diversity weighting alone, and show that encoder choice strongly modulates collapse severity (cosine 0.908 with mxbai versus 0.888 with nomic) and downstream accuracy. The more robust finding is that collapse is measurable, worsens on harder tasks, and that the choice of embedding proxy is a first-order design decision for any latent communication protocol.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。