arXiv:2603.05659cs.CVcs.AI2026-03被引 1

不依赖理想答案,通过计数错误来提升虚拟试衣的生成质量

When Rubrics Fail: Error Enumeration as Reward in Reference-Free RL Post-Training for Virtual Try-On

  • 用错误数量加权评分替代传统评分标准,构建可校准的奖励信号
  • 在多变输出场景中,错误计数法比规则评分更稳定有效
  • 适合缺乏唯一正确答案的生成任务,如虚拟试衣、内容创作等

强化学习中基于可验证奖励(RLVR)和规则评分(RaR)在有明确正确性信号或主观评价任务中表现优异,但许多真实任务存在多个合理输出且无单一理想答案,现有方法在此类参考自由设置下失效。本文提出隐式错误计数(IEC),不再对比回答与规则的一致性,而是统计错误并按严重性加权,转化为每方面的校准奖励。发现显式计数噪声过大,需采用隐式评分与分组校准设计才能保证优化稳定。以虚拟试衣(VTO)为例,该任务既不适合整体评分又不适合规则评估:细微服装错误不可接受,但多种输出均有效。引入级联错误计数(CEC)作为评估指标,与人类偏好高度一致(60%胜出率)。构建最大属性错配基准MDressBench用于测试奖励设计。在该基准上,IEC全面超越RaR(平坦参考:5.31 vs. 5.60;非平坦:5.20 vs. 5.53)。在VITON-HD和DressCode上,IEC在8个感知指标中的6项上达到或超过6个基线。

原文摘要 · Abstract (English)

Reinforcement learning with verifiable rewards (RLVR) and Rubrics as Rewards (RaR) have driven strong gains in domains with clear correctness signals and even in subjective domains by synthesizing evaluation criteria from ideal reference answers. But many real-world tasks admit multiple valid outputs and lack the single ideal answer that rubric generation depends on. We identify this reference-free setting as a gap in current post-training methods and propose Implicit Error Counting (IEC) to fill it. Instead of checking what a response gets right against a rubric, IEC enumerates what it gets wrong, applying severity-weighted scores across task-relevant axes and converting them into calibrated per-aspect rewards. We show that naïve explicit enumeration is too noisy for stable optimization, and that two design choices: implicit score emission and group calibration are necessary to make error counting a reliable reward. As a case study, we validate IEC on virtual try-on (VTO), a domain that is simultaneously too constrained for holistic scoring and too permissive for rubric-based evaluation: subtle garment errors are unacceptable, yet many output variations are correct. We introduce Cascaded Error Counting (CEC) as an evaluation metric, which tracks human preferences well (60% top-1 vs. 30% others), and curate Mismatch-DressCode (MDressBench), a benchmark with maximal attribute mismatch to stress-test reward designs. On MDressBench, IEC outperforms RaR across all metrics (CEC: 5.31 vs. 5.60 on flat references; 5.20 vs. 5.53 on non-flat). On VITON-HD and DressCode, IEC matches or surpasses six baselines on 6 of 8 perceptual metrics. These results suggest that when ideal answers are unavailable, counting errors provide a stronger signal than constructing rubrics.

虚拟试衣强化学习错误计数无参考训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。