用谱几何与随机矩阵理论，实现大模型实时幻觉检测与高效压缩。
Spectral Geometry for Deep Learning: Compression and Hallucination Detection via Random Matrix Theory
- 基于隐藏层激活的谱特征分析，捕捉模型运行时的动态变化。
- 可实时检测语言模型幻觉，且在压缩后仍保持高准确率。
- 适合关注模型可靠性与部署效率的研究者与工程师。
大型语言模型和深度神经网络虽表现强劲，但存在可靠性差与计算成本高的问题。本论文提出一种基于谱几何与随机矩阵理论的统一框架，通过分析隐藏层激活的特征值结构来解决上述问题。首个贡献是EigenTrack，一种实时检测语言与视觉-语言模型幻觉及分布外行为的方法，利用谱特征及其时间动态性。第二个贡献是RMT-KD，一种基于原理的压缩方法，能识别有意义的谱成分，并通过迭代知识蒸馏生成紧凑高效的模型，同时保持精度。结果表明，谱统计量可为大规模神经网络提供可解释且鲁棒的不确定性监测信号与压缩指导。
原文摘要 · Abstract (English)
Large language models and deep neural networks achieve strong performance but suffer from reliability issues and high computational cost. This thesis proposes a unified framework based on spectral geometry and random matrix theory to address both problems by analyzing the eigenvalue structure of hidden activations. The first contribution, EigenTrack, is a real-time method for detecting hallucinations and out-of-distribution behavior in language and vision-language models using spectral features and their temporal dynamics. The second contribution, RMT-KD, is a principled compression method that identifies informative spectral components and applies iterative knowledge distillation to produce compact and efficient models while preserving accuracy. Together, these results show that spectral statistics provide interpretable and robust signals for monitoring uncertainty and guiding compression in large-scale neural networks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。