用低秩适配降低物理信息深度算子网络的参数量，提升精度与泛化能力。
Low-rank adaptive physics-informed HyperDeepONets for solving differential equations
- 通过低秩分解压缩超网络输出层，减少参数量
- 在常微分方程和偏微分方程上实现最高70%参数缩减
- 适合需要高效高精度求解微分方程的科研与工程应用
HyperDeepONets 是一种用于算子学习的新型架构，由超网络生成 DeepONet 的主干网络权重。尽管表达能力强，但其高计算与存储开销源于大量输出参数。本文提出 PI-LoRA-HyperDeepONets，结合低秩适应（LoRA）技术，在物理信息机器学习框架下将超网络输出层权重矩阵分解为两个低秩矩阵，显著降低可训练参数数量，并对主干网络权重引入额外正则化。在常微分方程和偏微分方程上的大量实验表明，该方法可实现最高70%的参数减少，且预测精度和泛化能力持续优于传统 HyperDeepONets。
原文摘要 · Abstract (English)
HyperDeepONets were introduced in Lee, Cho and Hwang [ICLR, 2023] as an alternative architecture for operator learning, in which a hypernetwork generates the weights for the trunk net of a DeepONet. While this improves expressivity, it incurs high memory and computational costs due to the large number of output parameters required. In this work we introduce, in the physics-informed machine learning setting, a variation, PI-LoRA-HyperDeepONets, which leverage low-rank adaptation (LoRA) to reduce complexity by decomposing the hypernetwork's output layer weight matrix into two smaller low-rank matrices. This reduces the number of trainable parameters while introducing an extra regularization of the trunk networks' weights. Through extensive experiments on both ordinary and partial differential equations we show that PI-LoRA-HyperDeepONets achieve up to 70\% reduction in parameters and consistently outperform regular HyperDeepONets in terms of predictive accuracy and generalization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。