用物理原理训练的模型,能零微调跨模态迁移,性能优于传统方法。
Building The Ph(ysical)AI Layer Of Machine Intelligence

- 基于傅里叶分解、能量守恒等物理原则设计模型,避免盲目学统计关联。
- 1.99M参数模型在15个任务上平均准确率77.7%,物理类任务达84.5%。
- 适合需要高效跨模态迁移的物理建模与边缘智能场景。
基础模型通过大规模多样化数据训练实现泛化,但在无配对数据的情况下难以迁移到真正未见领域。我们提出基于原理的基础模型,将信号理论中的原则(傅里叶分解、能量守恒、对称性)编码,而非学习脱离物理的统计相关性。假设不同领域并非本质差异,而是时间、频率、幅度或相位上的可学习变换。仅在射频(RF)数据上训练,结合协同设计的架构与损失函数,实现对音频、图像、文本和视频的跨模态迁移,仅需冻结已学表示,无需在目标域微调编码器。1.99M参数的冻结编码器通过线性探测在15项任务上达到77.7%平均准确率(91.9% top-3),系统性表现:物理相关任务(说话人识别、地震学、射频指纹)达84.5%,语义任务(音乐流派、语言识别)为70.0%。这表明基于原理与基于规模的方法具有互补性:物理原则促进高效跨模态迁移,并自然划定物理与语义理解的边界。
原文摘要 · Abstract (English)
Foundation models achieve generalization through massive-scale training on diverse data, but have limitations with transfer to truly unseen domains without paired training data. We propose principle-driven foundation models that encode signal-theoretic principles (Fourier decomposition, energy conservation, symmetry) rather than learn untethered statistical correlations. We hypothesize that domains differ not in fundamental physics, but in learnable transformations in time, frequency, magnitude, or phase. Training exclusively on radio-frequency (RF) data with co-designed architecture and losses incorporating these principles, we achieve cross-modal transfer to audio, images, text, and video using only frozen representations learned from RF data, requiring no fine-tuning of the encoder on target domains. Our 1.99M parameter frozen encoder achieves 77.7% average accuracy (91.9% top-3) across 15 diverse tasks via linear probing, with systematic variation: 84.5 on physically-grounded tasks (speaker recognition, seismology, RF fingerprinting) versus 70.0% on semantic tasks (music genre, language recognition). This reveals that principle-driven and scale-driven approaches offer complementary paths: physical principles enable efficient cross-modal transfer while naturally establishing the boundary between physical and semantic understanding.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。