用区域化偏好优化提升文字渲染的字形准确性。
GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Rendering
- 基于区域级偏好标注,改进标准偏好优化方法。
- 在复杂字符上字形准确率显著提升,优于现有方法。
- 适合需要高精度字形输出的文本渲染场景。
生成准确的字形对于视觉文本渲染至关重要但极具挑战。现有方法通常依赖大量高质量场景文本图像进行训练,但字形变化覆盖有限且过度风格化常损害字形准确性,尤其在复杂或域外字符上。部分方法采用强化学习缓解此问题,但其奖励模型依赖对细粒度字形错误不敏感的文本识别系统,导致含错误字形的图像仍获高奖励。受直接偏好优化(DPO)启发,我们提出GlyphPrinter,一种无需显式奖励模型的偏好式文本渲染方法。然而,标准DPO仅建模样本间整体偏好,不足以应对视觉文本渲染中局部区域的字形错误。为此,我们构建了具有区域级字形偏好标注的GlyphCorrector数据集,并提出区域分组DPO(R-GDPO),通过优化标注区域间的跨样本与内样本偏好,显著提升字形准确性。此外,我们引入区域奖励引导策略,在推理时从可控字形准确率的最优分布采样。大量实验表明,所提GlyphPrinter在保持良好风格化与精度平衡的同时,显著优于现有方法。
原文摘要 · Abstract (English)
Generating accurate glyphs for visual text rendering is essential yet challenging. Existing methods typically enhance text rendering by training on a large amount of high-quality scene text images, but the limited coverage of glyph variations and excessive stylization often compromise glyph accuracy, especially for complex or out-of-domain characters. Some methods leverage reinforcement learning to alleviate this issue, yet their reward models usually depend on text recognition systems that are insensitive to fine-grained glyph errors, so images with incorrect glyphs may still receive high rewards. Inspired by Direct Preference Optimization (DPO), we propose GlyphPrinter, a preference-based text rendering method that eliminates reliance on explicit reward models. However, the standard DPO objective only models overall preference between two samples, which is insufficient for visual text rendering where glyph errors typically occur in localized regions. To address this issue, we construct the GlyphCorrector dataset with region-level glyph preference annotations and propose Region-Grouped DPO (R-GDPO), a region-based objective that optimizes inter- and intra-sample preferences over annotated regions, substantially enhancing glyph accuracy. Furthermore, we introduce Regional Reward Guidance, an inference strategy that samples from an optimal distribution with controllable glyph accuracy. Extensive experiments demonstrate that the proposed GlyphPrinter outperforms existing methods in glyph accuracy while maintaining a favorable balance between stylization and precision.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。