让视觉特征更难被还原，同时保持识别能力。
TrustCLIP: Learning Private Visual Features via Adversarial Reconstruction
- 用生成对抗方式训练特征投影，直接降低图像重建精度。
- 在分类和多模态大模型中，重建质量下降但任务性能不变。
- 针对生成式逆向攻击设计，比传统隐私评估更贴近真实威胁。
视觉与视觉语言模型依赖高层视觉表征，广泛用于识别、检索和多模态推理。然而，生成建模进展表明这些特征常可被反演，实现高保真图像重建，引发严重隐私风险。本文从重建视角出发，提出TrustCLIP——一种以重建为导向的防御框架，将特征条件生成器视为显式的隐私攻击者。TrustCLIP学习编码器特征与下游模块间的投影，显式优化以降低生成攻击者产生的重建质量，同时保留下游任务所需信号。不同于依赖判别式隐私度量的先前方法，TrustCLIP直接对抗生成式重建攻击，瞄准标准评估协议未涵盖的威胁。我们在常规分类和多模态大语言模型管道中验证其有效性：在各类设置下，信任剪枝均显著降低生成逆向的保真度，同时维持下游任务性能。项目页：https://atnikos.github.io/trustclip/
原文摘要 · Abstract (English)
Vision and vision-language models rely on high-level visual representations that are increasingly used across recognition, retrieval, and multimodal reasoning pipelines. However, recent advances in generative modeling have shown that such features can often be inverted, enabling realistic reconstructions of the underlying image and raising significant privacy risks. We revisit this problem through the lens of reconstruction and propose TrustCLIP, a reconstruction-driven framework that treats a feature-conditioned generator as an explicit privacy adversary. TrustCLIP learns a projection between encoder features and downstream modules that is explicitly optimized to degrade the reconstructions produced by generative attackers while retaining the necessary signals for downstream tasks. Unlike prior defenses that rely on discriminative privacy metrics, TrustCLIP directly optimizes against a generative reconstruction attacker, targeting a threat not captured by standard evaluation protocols. We demonstrate its effectiveness in both conventional classification and multimodal large language model pipelines. Across these settings, TrustCLIP consistently reduces the fidelity of generative inversions while maintaining downstream task performance. Project page: https://atnikos.github.io/trustclip/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。