通过频域分析参数轨迹,提升模型在复杂时间漂移下的泛化能力。
Learning Robust Spectral Dynamics for Temporal Domain Generalization
- 将参数变化分解为频谱成分,低频用Koopman算子预测,高频用正则化平滑。
- 在真实流数据场景中显著优于现有SOTA方法,尤其适应周期性与突发漂移。
- 兼具理论保障与实证性能,适合应对动态环境中的持续学习任务。
现代机器学习模型在存在时间分布漂移(即概念漂移)的动态环境中难以保持性能。时间域泛化(TDG)旨在实现跨演化域的模型泛化,但现有方法通常假设变化是平滑渐进的,难以应对包含长期结构(如渐进演化或周期性)和局部不确定性的复杂现实漂移。为此,本文提出FreKoo,通过参数轨迹的新型频域分析解决上述挑战。该方法利用傅里叶变换将参数演化解耦为不同频带成分:低频主成分采用Koopman算子学习并外推,有效捕捉增量型与周期性漂移模式;同时,潜在扰动的高频成分通过针对性的时间正则化加以抑制,防止对瞬时噪声和域不确定性的过拟合。该双谱策略经严格理论分析支撑,提供Koopman预测的稳定性保证、高频正则化的贝叶斯合理性,并导出连接频谱动态与泛化性能的多尺度泛化界。大量实验表明,FreKoo在复杂漂移与不确定性的真实流数据场景中显著优于当前最优的TDG方法。
原文摘要 · Abstract (English)
Modern machine learning models struggle to maintain performance in dynamic environments where temporal distribution shifts, \emph{i.e., concept drift}, are prevalent. Temporal Domain Generalization (TDG) seeks to enable model generalization across evolving domains, yet existing approaches typically assume smooth incremental changes, struggling with complex real-world drifts involving long-term structure (incremental evolution/periodicity) and local uncertainties. To overcome these limitations, we introduce FreKoo, which tackles these challenges via a novel frequency-domain analysis of parameter trajectories. It leverages the Fourier transform to disentangle parameter evolution into distinct spectral bands. Specifically, low-frequency component with dominant dynamics are learned and extrapolated using the Koopman operator, robustly capturing diverse drift patterns including both incremental and periodicity. Simultaneously, potentially disruptive high-frequency variations are smoothed via targeted temporal regularization, preventing overfitting to transient noise and domain uncertainties. In addition, this dual spectral strategy is rigorously grounded through theoretical analysis, providing stability guarantees for the Koopman prediction, a principled Bayesian justification for the high-frequency regularization, and culminating in a multiscale generalization bound connecting spectral dynamics to improved generalization. Extensive experiments demonstrate FreKoo's significant superiority over SOTA TDG approaches, particularly excelling in real-world streaming scenarios with complex drifts and uncertainties.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。