提出可学习的切尔诺夫基线，高效实现生成模型推理对齐。
Learnable Chernoff Baselines for Inference-Time Alignment
- 基于自适应接受概率的拒绝采样，仅需黑盒采样访问
- 在连续与离散扩散模型中显著减少预训练模型调用次数
- 提供变分距离保证，适合需要高效推理的部署场景
我们研究生成模型在推理阶段的奖励引导对齐问题。现有方法通常依赖特定架构或计算开销大的推理过程。本文提出可学习的切尔诺夫基线（LCBs），用于高效近似采样来自KL正则化奖励对齐的指数倾斜核。仅需对预训练模型进行黑盒采样访问，LCBs采用自适应选择接受概率的拒绝采样方式，实现对推理-计算成本的细粒度控制。我们建立了与理想对齐模型之间的总变差界，并在连续与离散扩散设置中验证，LCB采样能紧密逼近理想拒绝采样，同时大幅减少对预训练模型的查询次数。
原文摘要 · Abstract (English)
We study inference-time reward-guided alignment for generative models. Existing methods often rely on either architecture-specific adaptations or computationally costly inference procedures. We introduce Learnable Chernoff Baselines (LCBs) as a method for efficiently and approximately sampling from the exponentially tilted kernels that arise from KL-regularized reward alignment. Using only black-box sampling access to the pretrained model, LCBs implement a form of rejection sampling with adaptively selected acceptance probabilities, which allows fine-grained control over inference-compute scaling. We establish total-variation guarantees to the ideal aligned model, and demonstrate in both continuous and discrete diffusion settings that LCB sampling closely matches ideal rejection sampling while using substantially fewer queries to the pretrained model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。