arXiv:2607.14103cs.CLcs.MA2026-07被引 1

用隐空间通信可压缩99%信息,但文本丢失88%关键特征。

Latent Communication Between Language Model Agents: Channels, Alignment, and the Limits of Text

论文配图:Latent Communication Between Language Model Agents: Channels, Alignment, and the Limits of Text
图 1 · 摘自论文原文
  • 构建三种通信通道,用稀疏自编码器分析信息保留率
  • 隐空间通道压缩28倍仍保99.4%准确率,文本通道仅80.4%
  • 文本序列化摧毁88%特征,主要因表层形式替换而非衰减

多智能体系统依赖文本消息传递进行通信。我们假设大语言模型具备超越文本表达能力的世界模型,用于传递复杂概念。通过结构化实验验证该假设:使用稀疏自编码器(SAE)分析发现,文本通信存在信息损失。构建三种通信通道并测量概念区分信息量:在28倍压缩下,SAE稀疏通道保持99.4%探测准确率,而文本通道仅为80.4%。跨架构通信中,经Procrustes对齐后Llama与Mistral间顶1检索率达92%。通过文本往返测试发现,文本序列化导致88%的SAE特征消失,被另一组特征取代,归因于身份替换而非衰减。线性对齐带来3-10个百分点性能损失,非线性方法改善效果。任务层面评估显示,隐空间通道在跨语言概念任务上与文本通道相当,但未超越;引入隐特征增广也未提升性能。结论:丢失特征大多编码表面形式,而非任务相关语义,因此当前证据不支持隐空间通信优于文本的假设。

原文摘要 · Abstract (English)

Multi-agent systems (MAS) are utilized in many contexts and many professions. Those MAS rely on inter-agent communication, usually implemented by clear-text message passing. We hypothesize that Large Language Models may have a world model at their disposal that exceeds expressibility in text when complex concepts need to be communicated. Our aim is to approach a proof of this hypothesis with structured experiments. In this work, we show that LLM agents communicating via text lose information, which we quantify via Sparse Autoencoder (SAE) feature analysis. We construct three communication channels and measure concept-discriminating information in each. We first show that the SAE-sparse channel retains a 99.4% probe accuracy at 28-fold compression over the dense-latent channel vs 80.4% for the text channel. We then proceed to examine the same for cross-architecture communication by using sparse latent space alignment. We find for Procrustes alignment a 92% top-1 retrieval between Llama and Mistral. Using a text round-trip, we perform feature survival analysis to find that text serialization destroys 88% of SAE features, replacing them with a different feature set. We attribute the loss to identity replacement, not attenuation. By our analysis, we were able to attribute a 3-10pp performance penalty to the linear Procrustes alignment, improving with nonlinear alignment methods. In a task-level evaluation we find that the latent channel matches the text channel on cross-lingual concept tasks but never exceeds it. Text augmentation with latent features provides no benefit, leading us to negative conclusions for the initial hypothesis: lost features mostly or completely encode surface form, not task-relevant semantics. To pinpoint the practical advantage of latent communication over a text channel, deeper tasks eliciting complex concepts and an corresponding analysis framework are needed.

LLM智能体隐空间通信信息压缩特征分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。