用双曲几何建模模型演化，实现兼容更新而不重算旧数据。
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
- 将嵌入升维到双曲空间，用时间轴表示模型置信度变化
- 新旧模型保持生成一致性，避免劣质旧表示拖累新模型
- 动态调整对齐权重，适应旧模型不确定性，适合持续学习场景
后向兼容的表示学习使更新后的模型能无缝接入已有系统，无需重新处理存储数据。尽管已有进展，现有欧氏空间方法忽视旧嵌入模型的不确定性，强制新模型重建过时表示，阻碍学习过程。本文提出转向双曲几何，将时间视为自然轴以捕捉模型置信度与演化过程。通过将嵌入提升至双曲空间，并约束更新后的嵌入位于旧嵌入的蕴含锥内，我们在保持模型代际一致性的同时，考虑了表示中的不确定性。为进一步增强兼容性，引入一种鲁棒的对比对齐损失，根据旧嵌入的不确定性动态调整对齐权重。实验验证了该方法在实现兼容性方面的优越性，为更稳健、可适应的机器学习系统铺平道路。
原文摘要 · Abstract (English)
Backward compatible representation learning enables updated models to integrate seamlessly with existing ones, avoiding to reprocess stored data. Despite recent advances, existing compatibility approaches in Euclidean space neglect the uncertainty in the old embedding model and force the new model to reconstruct outdated representations regardless of their quality, thereby hindering the learning process of the new model. In this paper, we propose to switch perspectives to hyperbolic geometry, where we treat time as a natural axis for capturing a model's confidence and evolution. By lifting embeddings into hyperbolic space and constraining updated embeddings to lie within the entailment cone of the old ones, we maintain generational consistency across models while accounting for uncertainties in the representations. To further enhance compatibility, we introduce a robust contrastive alignment loss that dynamically adjusts alignment weights based on the uncertainty of the old embeddings. Experiments validate the superiority of the proposed method in achieving compatibility, paving the way for more resilient and adaptable machine learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。