用信息几何优化软最大分布的语义编码，实现精准概念操控。
The Information Geometry of Softmax: Probing and Steering
- 以信息几何为框架，揭示软最大分布的自然几何结构
- 提出双线性操控方法，精准改变目标概念且干扰最小
- 提升概念操控的稳定性和可控性，适合可解释性研究
本文探讨人工智能系统如何将语义结构编码到表征空间的几何结构中。核心观察是：表征空间的自然几何应反映模型利用表征产生行为的方式。研究聚焦于定义软最大分布的表征情形，认为此时自然几何为信息几何。重点分析信息几何在语义编码与线性表征假设中的作用。作为应用，提出“双线性操控”方法，通过线性探测器稳健地引导表征体现特定概念。理论证明该方法能最优地修改目标概念，同时最小化对非目标概念的影响。实验表明，双线性操控显著提升了概念操控的可控性与稳定性。
原文摘要 · Abstract (English)
This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation is that the natural geometry of these representation spaces should reflect the way models use representations to produce behavior. We focus on the important special case of representations that define softmax distributions. In this case, we argue that the natural geometry is information geometry. Our focus is on the role of information geometry on semantic encoding and the linear representation hypothesis. As an illustrative application, we develop "dual steering", a method for robustly steering representations to exhibit a particular concept using linear probes. We prove that dual steering optimally modifies the target concept while minimizing changes to off-target concepts. Empirically, we find that dual steering enhances the controllability and stability of concept manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。