用热带几何证明稀疏性本质是组合深度，解释MoE为何高效。
Sparsity is Combinatorial Depth: Quantifying MoE Expressivity via Tropical Geometry
- 首次用热带几何分析MoE，发现路由机制等价于第k个初等对称热带多项式。
- 稀疏性使表达能力提升至组合数$\binom{N}{k}$倍，突破传统容量限制。
- 揭示共享专家必要性，为MoE架构设计提供理论依据，适合模型优化研究者。
尽管混合专家（MoE）架构处于前沿地位，但其理论成功常被归因于启发式效率而非几何表达能力。本文首次通过热带几何视角分析MoE，证明Top-$k$路由机制在代数上同构于第$k$个初等对称热带多项式。该同构将输入空间划分为超单纯形的法锥图，揭示了‘稀疏性即组合深度’，其几何容量按二项式系数$\binom{N}{k}$增长。超越环境边界，我们提出在流形假设下的‘有效容量’概念。证明密集网络在低维数据上会遭遇容量坍缩，而MoE凭借路由锥的横截性表现出‘组合鲁棒性’，维持高表达能力。基于此理论界，我们推导出最优专家粒度的渐近容量极限，并证明共享专家在几何上是防止路由坍缩的必要条件。
原文摘要 · Abstract (English)
While Mixture-of-Experts (MoE) architectures define the state-of-the-art, their theoretical success is often attributed to heuristic efficiency rather than geometric expressivity. In this work, we present the first analysis of MoE through the lens of tropical geometry, establishing that the Top-$k$ routing mechanism is algebraically isomorphic to the $k$-th elementary symmetric tropical polynomial. This isomorphism partitions the input space into the Normal Fan of a Hypersimplex, revealing that \textbf{sparsity is combinatorial depth} which scales geometric capacity by the binomial coefficient $\binom{N}{k}$. Moving beyond ambient bounds, we introduce the concept of \textit{Effective Capacity} under the Manifold Hypothesis. We prove that while dense networks suffer from capacity collapse on low-dimensional data, MoE architectures exhibit \textit{Combinatorial Resilience}, maintaining high expressivity via the transversality of routing cones. Translating these theoretical bounds into architectural principles, we derive asymptotic capacity limits for optimal expert granularity and prove that shared experts are geometrically necessary to prevent routing collapse.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。