arXiv:2507.18996cs.LGstat.ML2025-07

基于费舍尔信息设计自适应框架,应对数据分布动态变化。

Adapting to Fragmented and Evolving Data: A Fisher Information Perspective

  • 用费舍尔信息引导参数更新,根据敏感度和稳定性调节学习。
  • 在七组基准上,严重分布偏移下准确率提升最高达19%。
  • 无需标签或任务边界,适合在线学习与联邦学习场景。

现代机器学习系统在动态环境中常面临序列协变量偏移(SCS),即输入分布随时间演变而条件分布保持稳定。本文提出轻量级且理论严谨的FADE(基于费舍尔信息的动态环境自适应)框架,通过结合费舍尔信息几何的感知正则化机制,依据参数敏感性与稳定性调控更新过程。为检测显著分布变化,提出受克雷默-罗不等式启发的偏移信号,融合KL散度与时间维度上的费舍尔动力学。相比以往方法需任务边界、目标监督或经验回放,FADE支持在线学习、固定内存且无需目标标签。在涵盖视觉、语言与表格数据的七个基准上评估,于严重偏移下最高提升19%准确率,优于TENT与DIW等方法。该框架可自然推广至联邦学习,将异构客户端视为时序碎片化环境,实现去中心化场景下的可扩展稳定自适应。理论分析保证有界遗憾与参数一致性,实证结果验证其跨模态与偏移强度的鲁棒性。

原文摘要 · Abstract (English)

Modern machine learning systems operating in dynamic environments often face \textit{sequential covariate shift} (SCS), where input distributions evolve over time while the conditional distribution remains stable. We introduce FADE (Fisher-based Adaptation to Dynamic Environments), a lightweight and theoretically grounded framework for robust learning under SCS. FADE employs a shift-aware regularization mechanism anchored in Fisher information geometry, guiding adaptation by modulating parameter updates based on sensitivity and stability. To detect significant distribution changes, we propose a Cramer-Rao-informed shift signal that integrates KL divergence with temporal Fisher dynamics. Unlike prior methods requiring task boundaries, target supervision, or experience replay, FADE operates online with fixed memory and no access to target labels. Evaluated on seven benchmarks spanning vision, language, and tabular data, FADE achieves up to 19\% higher accuracy under severe shifts, outperforming methods such as TENT and DIW. FADE also generalizes naturally to federated learning by treating heterogeneous clients as temporally fragmented environments, enabling scalable and stable adaptation in decentralized settings. Theoretical analysis guarantees bounded regret and parameter consistency, while empirical results demonstrate FADE's robustness across modalities and shift intensities.

自适应学习分布偏移费舍尔信息联邦学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。