arXiv:2409.07953cs.LG2024-09被引 31

揭示张量分解与电路表示的深层联系,统一建模方法并支持高效架构设计。

What is the Relationship between Tensor Factorizations and Circuits (and How Can We Exploit it)?

  • 用模块化积木思想构建可扩展的张量化电路架构
  • 在多个数据集上验证框架能有效提升模型表达能力
  • 适合对张量计算与概率建模交叉研究者参考

本文建立了电路表示与张量分解之间的严格关联,这两个看似不同但本质相关的领域。通过连接两者,我们揭示了对双方社区均有价值的一系列新机遇。工作将主流张量分解方法推广至电路语言,并将多种电路学习算法统一到一个广义分层分解框架下。具体提出一种模块化的‘乐高积木’方法,用于构建张量化电路架构,从而系统性地构造和探索各类电路与张量分解模型,同时保持可计算性。该连接不仅厘清了现有模型间的异同,还推动了构建与优化新型电路/张量分解架构的完整流程。通过大量实证评估验证了框架的有效性,并指出张量分解在概率建模中的新研究方向。

原文摘要 · Abstract (English)

This paper establishes a rigorous connection between circuit representations and tensor factorizations, two seemingly distinct yet fundamentally related areas. By connecting these fields, we highlight a series of opportunities that can benefit both communities. Our work generalizes popular tensor factorizations within the circuit language, and unifies various circuit learning algorithms under a single, generalized hierarchical factorization framework. Specifically, we introduce a modular "Lego block" approach to build tensorized circuit architectures. This, in turn, allows us to systematically construct and explore various circuit and tensor factorization models while maintaining tractability. This connection not only clarifies similarities and differences in existing models, but also enables the development of a comprehensive pipeline for building and optimizing new circuit/tensor factorization architectures. We show the effectiveness of our framework through extensive empirical evaluations, and highlight new research opportunities for tensor factorizations in probabilistic modeling.

张量分解电路表示建模框架

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。