提出面向文本生成SVG的对齐人类判断的评估框架,解决现有方法不准确的问题。
SVG-Score: Human-Aligned Evaluation of Text-to-SVG Generation

- 基于可控扰动实验,发现现有评估指标对颜色、数量等错误不敏感。
- 构建人工标注的语义对齐数据集,用于训练更符合人类判断的评估模型。
- 开发快速评估与可解释评估双引擎,适配不同场景的SVG生成评测需求。
随着生成模型在表达力和可控性上的提升,可缩放矢量图形(SVG)生成日益受到关注。然而,领域特定的评估协议缺失制约了进展:当前普遍采用为自然图像设计的指标(如CLIPScore),其从未在矢量图形上训练,与人类判断仅部分对齐。本文提出 extbf{ ous},一个面向文本到SVG生成的人类对齐评估框架。通过控制性标题与图像扰动实验,我们发现基于CLIP的评分几乎不响应生成器实际犯下的错误,如颜色错、数量错、空间关系错;而通用视觉语言模型(VLM)虽更敏感,但对不同错误类型和风格反应不均。在此基础上,我们构建了人工标注的语义对齐数据集,衡量生成SVG与标题的忠实度。基于此,开发两类互补评估器:经向量图形适配并校准至人类偏好的CLIP评分器,用于大规模快速评估;以及通过监督微调与奖励驱动强化学习训练的VLM判别器,实现更具表现力和可解释性的评估。利用二者,我们在独立标题集上对主流开源、商业及优化型SVG生成器进行了基准测试。
原文摘要 · Abstract (English)
Scalable Vector Graphics (SVG) generation is attracting increasing attention as generative models improve in expressiveness and controllability. Progress, however, is held back by the lack of domain-specific evaluation protocols: current practice relies on metrics designed for natural images, most notably CLIPScore, which was never trained on vector graphics and aligns only partially with human judgment. We introduce \textbf{\ours}, a human-aligned evaluation framework for text-to-SVG generation. Through controlled caption and image perturbations, we first show that CLIP-based scores barely react to the errors SVG generators actually make, such as wrong colors, counts, and spatial relations, and that off-the-shelf Vision-Language Model (VLM) judges, while more sensitive, respond unevenly across error types and SVG styles. We then introduce a human-annotated dataset for \textit{Semantic Alignment}, measuring how faithfully a generated SVG reflects its caption. Building on it, we develop two complementary evaluators: CLIP scorers adapted to vector graphics and then aligned to human preferences, for fast large-scale evaluation, and a VLM judge trained with supervised fine-tuning and reward-shaped reinforcement learning, for more expressive and interpretable assessment. Using both, we benchmark major open-source, commercial, and optimization-based SVG generators on an independent caption set.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。