用RGB视频重建高光谱数据，发现肉眼看不见的伪造痕迹。
HyperFake: Hyperspectral Reconstruction and Attention-Guided Analysis for Advanced Deepfake Detection
- 从普通RGB视频重建31通道高光谱数据，挖掘隐藏伪造特征。
- 在DeepfakeDetection、FaceForensics++等数据集上准确率达97.8%以上。
- 无需高成本设备，适合需要泛化能力的实战检测场景。
深度伪造对数字媒体安全构成重大威胁，现有检测方法在不同篡改技术和数据集间泛化能力不足。尽管近期方法结合卷积神经网络与视觉变压器或利用多模态学习，仍受限于RGB数据的固有局限。我们提出HyperFake，一种全新的深度伪造检测流程：将标准RGB视频重建为31通道高光谱数据，揭示传统方法无法察觉的伪造痕迹。通过改进的MST++架构提升高光谱重建质量，并引入光谱注意力机制筛选关键波段特征用于检测。经优化的基于EfficientNet的分类器对重构光谱数据进行分析，实现跨不同伪造风格和数据集的高精度、强泛化检测，且无需依赖昂贵的高光谱相机。据我们所知，这是首个利用高光谱成像重建进行深度伪造检测的方法，为应对日益复杂的伪造手段开辟了新路径。
原文摘要 · Abstract (English)
Deepfakes pose a significant threat to digital media security, with current detection methods struggling to generalize across different manipulation techniques and datasets. While recent approaches combine CNN-based architectures with Vision Transformers or leverage multi-modal learning, they remain limited by the inherent constraints of RGB data. We introduce HyperFake, a novel deepfake detection pipeline that reconstructs 31-channel hyperspectral data from standard RGB videos, revealing hidden manipulation traces invisible to conventional methods. Using an improved MST++ architecture, HyperFake enhances hyperspectral reconstruction, while a spectral attention mechanism selects the most critical spectral features for deepfake detection. The refined spectral data is then processed by an EfficientNet-based classifier optimized for spectral analysis, enabling more accurate and generalizable detection across different deepfake styles and datasets, all without the need for expensive hyperspectral cameras. To the best of our knowledge, this is the first approach to leverage hyperspectral imaging reconstruction for deepfake detection, opening new possibilities for detecting increasingly sophisticated manipulations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。