提出一种基于渐近乐观性的张量回归秩选择方法，可用于神经网络压缩。
Asymptotic Optimism for Tensor Regression Models with Applications to Neural Network Compression
- 基于高斯设计推导出张量分解的训练-测试误差期望，揭示真实秩时乐观性最小。
- 在图像回归任务中验证方法有效性，可实现神经网络张量压缩。
- 无需交叉验证，适用于深度学习中的模型秩选择与平均集成。
研究随机协变量设计下低秩张量回归的秩选择问题。在高斯随机设计模型和一些温和条件下，推导了CP与Tucker分解的期望训练-测试差异（乐观性）的总体表达式。进一步证明，对于两种分解方式，真实张量秩处的乐观性最小。由此得到一种以预测为导向的秩选择规则，其效果等同于交叉验证，并自然扩展至张量模型平均。同时讨论了欠秩或过秩模型可能更优的条件，明确了方法适用范围。最后，在真实世界图像回归任务中展示其实际效用，并将其应用于基于张量的神经网络压缩，凸显其在深度学习模型选择中的潜力。
原文摘要 · Abstract (English)
We study rank selection for low-rank tensor regression under random covariates design. Under a Gaussian random-design model and some mild conditions, we derive population expressions for the expected training-testing discrepancy (optimism) for both CP and Tucker decomposition. We further demonstrate that the optimism is minimized at the true tensor rank for both CP and Tucker regression. This yields a prediction-oriented rank-selection rule that aligns with cross-validation and extends naturally to tensor-model averaging. We also discuss conditions under which under- or over-ranked models may appear preferable, thereby clarifying the scope of the method. Finally, we showcase its practical utility on a real-world image regression task and extend its application to tensor-based compression of neural network, highlighting its potential for model selection in deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。