针对物理模型压缩中精度与物理规律易失的问题,提出敏感度感知的压缩方法。
SAFE-SVD: Sensitivity-Aware Fidelity-Enforcing SVD for Physics Foundation Models

- 在输出函数空间中建模层敏感度,动态调整压缩策略。
- 多模型多数据集测试中实现数倍于现有方法的压缩比。
- 适合需要高保真物理模拟的科研与工程部署场景。
我们提出一种新型方法,用于压缩物理基础模型(PFMs),这是人工智能赋能科学的新趋势。尽管模型压缩对降低大模型内存占用和加速推理至关重要,但在强调物理保真的PFMs中仍研究不足。挑战在于物理数据具有函数特性,其偏导数编码时空动态,对压缩高度敏感。传统压缩方法忽略此结构,常导致性能严重下降甚至失败。为此,我们引入一种敏感度感知、保真度约束的压缩框架,在压缩过程中显式建模输出函数空间中的损失感知层敏感度。该方法为科学基础模型的高效压缩提供了新路径,同时保持精度与物理一致性。实验表明,在多个模型和数据集上显著优于现有方法,压缩比提升数倍,某些情况下达到数量级增长。本工作或可推动高效、可部署、可持续的科学基础模型这一新子领域的形成。
原文摘要 · Abstract (English)
We propose a new method for compressing physics foundation models (PFMs) which is a new trend in AI for Science. While model compression is essential for reducing memory use and accelerating inference in large foundation models, it remains under-explored for PFMs, where preserving physical fidelity is crucial. The challenge lies in the functional nature of physics data, where partial derivatives encode spatiotemporal dynamics and exhibit high sensitivity to compression. Conventional compression methods ignore this structure, often causing severe performance degradation or failure. To address this, we introduce a sensitivity-aware fidelity-enforcing compression framework that explicitly models loss-aware layer sensitivity in the output function space during compression. This provides a new route to compressing scientific foundation models while preserving accuracy and physical fidelity. Experiments show substantial gains over existing methods across multiple models and datasets, achieving significantly higher compression ratios while maintaining accuracy, in some cases by orders of magnitude. More broadly, the work potentially leads to a new subfield of efficient, deployable, and sustainable scientific foundation models in AI for Science.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。