arXiv:2605.25442cs.CV2026-05

用多模态大模型提升单图人脸去混淆,恢复原始人脸身份。

Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models

论文配图:Enhancing Single-Image Facial Demorphing using Multimodal Large Language Models
图 1 · 摘自论文原文
  • 利用多模态大模型中间层语义嵌入,指导扩散模型联合重建两张原始人脸。
  • 在严格评估点上,性能比潜空间方法高30%~40%,恢复细节更精准。
  • 无需参考图像或身份重叠,适合司法取证与防欺诈场景。

人脸识别系统正面临混合作弊攻击的威胁,即通过合成图像匹配多个身份,导致非法访问与身份盗用。现有检测方法仅能识别混杂图像,无法恢复原始成分或身份,限制了其法证价值。本文提出一种无需参考的面部去混淆框架,利用多模态大语言模型(MLLM)引导基于扩散模型的联合重建过程。关键创新在于从MLLM中间层提取语义嵌入,作为条件信号,提供关于面部属性和身份线索的高层推理,补充低层像素信息。我们将去混淆建模为联合条件生成问题,通过直接在RGB域运行的去噪扩散模型同步生成两个组成人脸,确保身份一致性并保留精细感知细节。相比依赖压缩潜空间或假设训练测试集身份重叠的方法,本方法跳过有损的文本生成-重编码循环,直接使用MLLM隐藏状态作为条件信号,使去噪网络能够关注发丝、背景和面部纹理等细微视觉线索。消融实验表明:中层MLLM层包含更具身份区分性的表征,基于RGB域的去混淆在严格评估点上比潜空间方法提升30%~40%,全量MLLM嵌入相比原始ViT特征具有显著优势,得益于多模态预训练带来的语义结构增强。

原文摘要 · Abstract (English)

Face recognition systems are increasingly vulnerable to morphing attacks, where a composite image is crafted to match multiple identities, enabling unauthorized access and identity fraud. Existing detection methods identify morphed images but cannot recover constituent images or identities, limiting their forensic utility. This paper presents a novel reference-free facial demorphing framework that leverages Multimodal Large Language Models (MLLMs) to guide a coupled diffusion-based reconstruction process. Our key innovation lies in extracting semantic embeddings from intermediate MLLM layers to condition the demorphing, providing high-level reasoning about facial attributes and identity cues that complement low-level pixel information. We formulate demorphing as a coupled conditional generation problem, where both constituent faces are synthesized jointly through a denoising diffusion model operating directly in the RGB domain, ensuring inter-identity consistency while preserving fine-grained perceptual details. Unlike prior approaches that rely on compressed latent representations or assume identity overlap between training and testing sets, our method bypasses lossy text generation-reencoding cycles by directly utilizing MLLM hidden states as conditioning signals, enabling the denoising network to attend to subtle visual cues such as hair, background, and facial textures. Ablation studies further reveal that middle MLLM layers encode more identity-discriminative representations, RGB-domain demorphing outperforms latent-space approaches by 30--40\% at strict operating points, and full MLLM embeddings provide substantial advantages over raw ViT features through enhanced semantic structuring from multimodal pretraining.

人脸去混淆多模态模型扩散模型身份安全

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。