通过傅里叶模型揭示量子神经网络训练性能与偏差的内在关系
Fisher Information, Training and Bias in Fourier Regression Models
- 利用傅里叶模型等价性分析有效维度与任务偏差的影响机制
- 无偏模型中高有效维度提升可训练性,有偏模型则反之
- 为量子机器学习及经典模型设计提供几何与任务对齐的指导
受量子机器学习特别是量子神经网络(QNN)日益增长的兴趣推动,本文研究基于费舍尔信息矩阵(FIM)的评估指标在预测其训练与预测性能方面的有效性。我们利用一大类QNN与傅里叶模型的等价性,探讨模型有效维度与对特定任务的偏差之间的相互作用,分析其如何影响训练与性能。结果表明:对于完全无偏的模型,更高的有效维度通常带来更好的可训练性和性能;而对目标函数有偏的模型,较低的有效维度在训练中更占优势。为此,我们推导出傅里叶模型的FIM解析表达式,并识别控制有效维度的关键特征,从而可构建具有可调有效维度和偏差的模型并进行训练比较。此外,我们引入了所考虑傅里叶模型的张量网络表示,该表示对QNN分析可能具有独立价值。总体而言,这些发现为几何特性、模型-任务对齐与训练之间的复杂关系提供了明确范例,对更广泛的机器学习社区具有参考意义。
原文摘要 · Abstract (English)
Motivated by the growing interest in quantum machine learning, in particular quantum neural networks (QNNs), we study how recently introduced evaluation metrics based on the Fisher information matrix (FIM) are effective for predicting their training and prediction performance. We exploit the equivalence between a broad class of QNNs and Fourier models, and study the interplay between the \emph{effective dimension} and the \emph{bias} of a model towards a given task, investigating how these affect the model's training and performance. We show that for a model that is completely agnostic, or unbiased, towards the function to be learned, a higher effective dimension likely results in a better trainability and performance. On the other hand, for models that are biased towards the function to be learned a lower effective dimension is likely beneficial during training. To obtain these results, we derive an analytical expression of the FIM for Fourier models and identify the features controlling a model's effective dimension. This allows us to construct models with tunable effective dimension and bias, and to compare their training. We furthermore introduce a tensor network representation of the considered Fourier models, which could be a tool of independent interest for the analysis of QNN models. Overall, these findings provide an explicit example of the interplay between geometrical properties, model-task alignment and training, which are relevant for the broader machine learning community.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。