提出可提取有序特征函数的新框架，实现更精准的特征重要性评估。
Eigenfunction Extraction for Ordered Representation Learning
- 基于模块化设计，实现对上下文核的谱分解与特征函数提取
- 在真实图像数据集上验证，恢复的特征值可作为有效特征选择指标
- 适合需要可解释性与高效表示的模型优化场景
近期表示学习进展表明，对比与非对比等常用目标隐式执行了由输入与其上下文关系诱导的上下文核的谱分解。然而，现有方法仅能恢复核的前几个特征函数的线性组合，而精确的谱分解对理解特征排序与重要性至关重要。本文提出一种通用框架，用于提取有序且可识别的特征函数，其模块化组件满足与上下文核兼容及现代场景可扩展性的关键要求。我们进一步说明，低秩近似与瑞利商优化两种主流方法均符合该框架。在合成核上的验证表明，所恢复的特征值可作为特征选择的有效重要性评分，在真实图像数据集上实现基于自适应维度表示的合理精度-效率权衡。
原文摘要 · Abstract (English)
Recent advances in representation learning reveal that widely used objectives, such as contrastive and non-contrastive, implicitly perform spectral decomposition of a contextual kernel, induced by the relationship between inputs and their contexts. Yet, these methods recover only the linear span of top eigenfunctions of the kernel, whereas exact spectral decomposition is essential for understanding feature ordering and importance. In this work, we propose a general framework to extract ordered and identifiable eigenfunctions, based on modular building blocks designed to satisfy key desiderata, including compatibility with the contextual kernel and scalability to modern settings. We then show how two main methodological paradigms, low-rank approximation and Rayleigh quotient optimization, align with this framework for eigenfunction extraction. Finally, we validate our approach on synthetic kernels and demonstrate on real-world image datasets that the recovered eigenvalues act as effective importance scores for feature selection, enabling principled efficiency-accuracy tradeoffs via adaptive-dimensional representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。