将神经网络转为张量形式,提升隐私与可解释性。
Tensorization of neural networks for improved privacy and interpretability
- 用采样点和黑箱函数构建张量列车表示
- 压缩模型时内存与速度平衡优于传统方法
- 适合保护训练数据特征、分析物理相变
我们提出一种张量化算法,用于构建函数的张量列车/矩阵乘积态(MPS)表示,基于随机投影和交叉插值思想。该方法仅需目标函数的黑箱访问权和少量定义关注域的采样点,特别适用于机器学习模型,其关注域自然由训练数据集定义。我们证明该方法可增强神经网络的隐私性和可解释性:(i)对编码训练数据分布模式的参数进行混淆;(ii)从MPS表示中高效估计物质的拓扑相。此外,该张量化可作为优化MPS的高效初始化方法,在模型压缩中相较传统方法实现更优的内存与时间复杂度权衡。
原文摘要 · Abstract (English)
We present a tensorization algorithm for constructing tensor train/matrix product state (MPS) representations of functions, drawing on sketching and cross interpolation ideas. The method only requires black-box access to the target function and a small set of sample points defining the domain of interest. Thus, it is particularly well-suited for machine learning models, where the domain of interest is naturally defined by the training dataset. We show that this approach can be used to enhance the privacy and interpretability of neural network models. Specifically, we apply our decomposition to (i) obfuscate neural networks whose parameters encode patterns tied to the training data distribution, and (ii) estimate topological phases of matter that are easily accessible from the MPS representation. Additionally, we show that this tensorization can serve as an efficient initialization method for optimizing MPS in general settings, and that, for model compression, our algorithm achieves a superior trade-off between memory and time complexity compared to conventional tensorization methods of neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。