对比视觉自回归模型在微调与隐私保护下的表现，发现其非隐私场景更优但隐私场景落后。
Implementing Adaptations for Vision AutoRegressive Model
- 针对视觉自回归模型设计多种微调策略并实证评估
- 非隐私微调下性能优于扩散模型，但隐私微调性能显著下降
- 首次系统研究该模型的差分隐私适应，适合隐私敏感图像生成研究者
视觉自回归模型（VAR）作为图像生成领域对扩散模型（DMs）的替代方案近期受到关注。本文聚焦于其适应性微调技术，旨在将预训练模型调整以执行特定下游任务，如医学图像生成。尽管扩散模型已有丰富微调方法，但针对视觉自回归模型的适配研究仍较少。此外，针对数据隐私保护的差分隐私（DP）适应在扩散模型中已广泛研究，而视觉自回归模型尚缺乏此类解决方案。本文实现并基准测试了多种适用于VAR的策略，并与最先进的扩散模型适应方法进行比较。结果表明，在非差分隐私条件下，VAR性能优于扩散模型；然而在差分隐私设置下，其性能明显下降，亟需进一步研究。代码已公开于 https://github.com/sprintml/finetuning_var_dp。
原文摘要 · Abstract (English)
Vision AutoRegressive model (VAR) was recently introduced as an alternative to Diffusion Models (DMs) in image generation domain. In this work we focus on its adaptations, which aim to fine-tune pre-trained models to perform specific downstream tasks, like medical data generation. While for DMs there exist many techniques, adaptations for VAR remain underexplored. Similarly, differentially private (DP) adaptations-ones that aim to preserve privacy of the adaptation data-have been extensively studied for DMs, while VAR lacks such solutions. In our work, we implement and benchmark many strategies for VAR, and compare them to state-of-the-art DM adaptation strategies. We observe that VAR outperforms DMs for non-DP adaptations, however, the performance of DP suffers, which necessitates further research in private adaptations for VAR. Code is available at https://github.com/sprintml/finetuning_var_dp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。