让生成模型精准满足指定分布要求，提升生成结果的可控性。
Calibrating Generative Models to Distributional Constraints
- 将校准问题转为带约束的优化，用近似损失替代难解约束
- 在上百个约束下显著降低偏差，支持百亿参数模型
- 适用于蛋白质设计、图像生成等需精确分布控制场景
生成模型常出现校准错误，即采样分布的统计特性（如某类生成物占比）偏离预期。本文将校准建模为约束优化问题，寻找在相对熵意义下最接近原模型且满足校准约束的分布。针对精确施加约束不可行的问题，提出两种微调替代目标：(1) 放松损失，将约束转化为校准偏差惩罚；(2) 奖励损失，将校准转化为奖励微调任务。实验表明，该方法在数百个并行约束下显著降低校准误差，适用于参数量达九亿的多种模型，涵盖蛋白质设计、图像生成与语言建模等应用。
原文摘要 · Abstract (English)
Generative models frequently suffer miscalibration, wherein statistics of the sampling distribution, such as the fraction of generations in a given class, deviate from desired values. We frame calibration as a constrained optimization problem and seek the closest model in Kullback-Leibler divergence satisfying a calibration constraint. To address the intractability of imposing these constraints exactly, we introduce two surrogate objectives for fine-tuning: (1) the relax loss, which replaces the constraint with a miscalibration penalty, and (2) the reward loss, which converts calibration into a reward fine-tuning problem. We demonstrate that these approaches substantially reduce calibration error across hundreds of simultaneous constraints and models with up to nine billion parameters, spanning applications in protein design, image generation, and language modeling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。