用双线性自编码器分解神经网络表示为二次多项式，实现可分析的非线性潜在结构。
Finding Manifolds With Bilinear Autoencoders
- 通过双线性自编码器将潜在表示分解为二次多项式。
- 提升重要性排序、聚类效果与激活稀疏性。
- 适合研究可解释性潜空间的算法开发者。
稀疏自编码器是揭示神经网络中可解释潜在表示的标准工具。然而，其解释依赖于输入，孤立研究不完整。多项式提供了解决方案：作为代数基本单元，可在不参考输入的情况下分析，并能描述从线性概念到复杂流形的各种结构。本文利用双线性自编码器高效地将表示分解为二次多项式。我们讨论了可诱导重要性排序、聚类和激活稀疏性的改进方法。这是通过代数性质实现非线性但可分析潜在空间的初步尝试。
原文摘要 · Abstract (English)
Sparse autoencoders are a standard tool for uncovering interpretable latent representations in neural networks. Yet, their interpretation depends on the inputs, making their isolated study incomplete. Polynomials offer a solution; they serve as algebraic primitives that can be analysed without reference to input and can describe structures ranging from linear concepts to complicated manifolds. This work uses bilinear autoencoders to efficiently decompose representations into quadratic polynomials. We discuss improvements that induce importance ordering, clustering, and activation sparsity. This is an initial step toward nonlinear yet analysable latents through their algebraic properties.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。