通过谱滤波提升高维数据局部内在维数估计的鲁棒性
Local Hessian Spectral Filtering for Robust Intrinsic Dimension Estimation
- 基于海森矩阵谱滤波，剔除法向大特征值，聚焦切向零曲率方向
- 在高维空间中显著优于现有方法，线性扩展至高维 $D$
- 适用于检测大规模扩散模型中的记忆现象，代码已开源
尽管扩散模型为局部内在维数（LID）估计提供了新思路，但现有方法在高维空间中因大量法向噪声掩盖切向信号而失效。本文提出局部海森谱维数（LHSD），通过在对数密度海森矩阵上应用谱滤波，显式剔除与法向方向相关的大型特征值，从而准确计数零曲率切向方向。采用随机兰佐斯二次型（SLQ）实现，避免完整海森矩阵构建，实现与维度 $D$ 的线性可扩展性。在合成数据和真实数据上的实验表明，LHSD 具有更强的鲁棒性，且能有效检测大规模扩散模型中的记忆现象。代码已发布于 github.com/geosada/LHSD。
原文摘要 · Abstract (English)
While diffusion models enable new approaches for estimating Local Intrinsic Dimension (LID), existing methods fail in high-dimensional spaces where noise from vast normal directions overwhelms the tangent signal. We propose Local Hessian Spectral Dimension (LHSD), which resolves this by applying spectral filtering to the log-density Hessian, explicitly cutting off large eigenvalues associated with normal directions to count zero-curvature tangent directions. Implemented using Stochastic Lanczos Quadrature (SLQ), LHSD avoids full Hessian construction, achieving linear scalability with dimension $D$. Experiments on synthetic and real data confirm LHSD's superior robustness and its utility in detecting memorization in large-scale diffusion models. The code is available at github.com/geosada/LHSD
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。