通过熵正则化轨迹实现信息瓶颈优化的稳定与凸性
Stable and Convexified Information Bottleneck Optimization via Symbolic Continuation and Entropy-Regularized Trajectories
- 引入熵正则化项，使信息瓶颈解路径保持凸性与唯一性
- 在多种β值下实现表示学习的稳定性，避免突变跳变
- 提供统计鲁棒的不确定性量化，适合理论与实际应用
信息瓶颈（IB）方法常因优化不稳而失效，表现为在信息瓶颈权衡参数β的临界点附近出现表示的剧烈突变。本文提出一种新方法，通过符号连续法与熵正则化轨迹，实现稳定且凸的IB优化。理论上证明了引入熵正则化后，IB解路径具有凸性与唯一性，并展示了该方法在广泛β取值下的表示学习稳定性。此外，本文还对临界点进行了详尽的敏感性分析，采用统计上稳健的不确定性量化（95%置信区间）。开源实现、实验结果及可复现框架为该方法的实际部署与未来扩展提供了清晰路径。
原文摘要 · Abstract (English)
The Information Bottleneck (IB) method frequently suffers from unstable optimization, characterized by abrupt representation shifts near critical points of the IB trade-off parameter, beta. In this paper, I introduce a novel approach to achieve stable and convex IB optimization through symbolic continuation and entropy-regularized trajectories. I analytically prove convexity and uniqueness of the IB solution path when an entropy regularization term is included, and demonstrate how this stabilizes representation learning across a wide range of \b{eta} values. Additionally, I provide extensive sensitivity analyses around critical points (beta) with statistically robust uncertainty quantification (95% confidence intervals). The open-source implementation, experimental results, and reproducibility framework included in this work offer a clear path for practical deployment and future extension of my proposed method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。