让个性化图像生成不泄露身份,还能自由调节隐私与画质的平衡。
IDDM: Identity-Decoupled Personalized Diffusion Models with a Tunable Privacy-Utility Trade-off

- 在微调过程中分离身份特征,防止生成图被识别人脸
- 实验显示能显著降低身份关联性,同时保持高质量输出
- 适合需要公开分享个性化内容但担心隐私泄露的用户
个性化文本到图像扩散模型(如DreamBooth、LoRA)可让用户用少量参考图生成高保真虚拟形象用于社交表达。然而,这些生成物一旦发布在社交媒体(如Instagram、Facebook),可能通过人脸识别系统被追踪到真实用户,造成身份暴露和画像分析。现有防御策略多采用反个性化方法,通过干扰微调过程来保护参考图,但无法应对授权个性化后仍存在身份泄露的问题。为此,本文提出新的防御场景——模型侧输出免疫,目标是在支持授权个性化的同时,降低公开生成物的身份关联性,并实现隐私与画质的可调平衡。为此,我们提出身份解耦式个性化扩散模型(IDDM),将身份解耦融入个性化流程。具体而言,IDDM采用交替优化策略,在短周期微调与身份解耦数据优化间循环,并使用两阶段调度机制平衡身份关联抑制与生成质量。在多个数据集、多样化提示词及主流人脸识别系统上的实验表明,IDDM持续降低身份关联性,同时保持高质量个性化生成。
原文摘要 · Abstract (English)
Personalized text-to-image diffusion models (e.g., DreamBooth, LoRA) enable users to synthesize high-fidelity avatars from a few reference photos for social expression. However, once these generations are shared on social media platforms (e.g., Instagram, Facebook), they can be linked to the real user via face recognition systems, enabling identity tracking and profiling. Existing defenses mainly follow an anti-personalization strategy that protects publicly released reference photos by disrupting model fine-tuning. While effective against unauthorized personalization, they do not address another practical setting in which personalization is authorized, but the resulting public outputs still leak identity information. To address this problem, we introduce a new defense setting, termed model-side output immunization, whose goal is to produce a personalized model that supports authorized personalization while reducing the identity linkability of public generations, with tunable control over the privacy-utility trade-off to accommodate diverse privacy needs. To this end, we propose Identity-Decoupled personalized Diffusion Models (IDDM), a model-side defense that integrates identity decoupling into the personalization pipeline. Concretely, IDDM follows an alternating procedure that interleaves short personalization updates with identity-decoupled data optimization, using a two-stage schedule to balance identity linkability suppression and generation utility. Extensive experiments across multiple datasets, diverse prompts, and state-of-the-art face recognition systems show that IDDM consistently reduces identity linkability while preserving high-quality personalized generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。