用物理一致性先验指导扩散模型修复交通标志,提升稀有标志数据生成质量。
Structured-Prior-Guided Diffusion Inpainting with Physical Consistency for Traffic Sign Augmentation

- 通过文本、颜色向量和几何模板三路注入符号先验,增强生成真实性。
- 在TT100K-2021上实现91.1%的OCR准确率,推理速度仅为大模型的1/14。
- 适合需要高保真交通标志合成的自动驾驶数据增强场景。
交通标志检测面临长尾数据分布问题,许多罕见标志虽样本极少但监管意义重大。通用图像修复模型直接应用于标志区域时会产生数字扭曲、几何变形和色彩偏移。本文归因于条件信号过于抽象,缺乏对标志物理结构的建模。为此提出结构化先验引导的扩散修复框架,通过三个独立路径注入语义、外观与几何先验:基于JSON的文本提示、由实测主色生成的前视图向量模板(IP-Adapter)、以及仿射对齐的矢量模板(ControlNet)。引入两个物理一致性损失:基于CIELAB色度的$ L_1 $项约束色彩,边缘梯度(Sobel)项约束结构。在自收集的大型内部数据集上进行自监督训练,零样本评估于公开的TT100K-2021数据集。采用约14亿参数的Stable Diffusion 1.5作为骨干,各项指标均超越7个基线模型。其OCR精确匹配率达91.1%,远超120亿参数工业模型FLUX.1 Fill[dev]的44.2%,且推理耗时仅为后者的1/14。消融实验表明三路先验与双损失均独立有效。下游检测任务中,合成数据使稀有类别组平均AP50提升1.23至7.40倍。代码与预训练模型已开源。
原文摘要 · Abstract (English)
Traffic sign detection faces a long-tailed data distribution. Many rare signs matter as much as common ones from a regulatory standpoint, yet they have very few samples. Generative data augmentation is one way out. General-purpose inpainting models, however, distort digits, deform geometry and perspective, and shift colours when applied directly to sign regions. We trace this to a single gap: the conditioning signal is too abstract for the physical composition of a sign. We propose a structured-prior-guided diffusion inpainting framework with physical consistency. It injects the semantic, appearance and geometric priors of a sign through three orthogonal pathways: a JSON-formatted text prompt, a front-view vector template rendered with measured dominant colours (via IP-Adapter), and an affine-aligned vector template (via ControlNet). Two physical consistency losses constrain colour with a CIELAB chromaticity $L_1$ term and edge structure with a Sobel gradient term. We train by self-supervised reconstruction on a large set of images collected in-house at AMAP, then evaluate zero-shot on the public TT100K-2021 dataset, a different source. Our method uses a Stable Diffusion 1.5 backbone of about 1.4B parameters. It beats seven representative competitors on every metric of reconstruction fidelity, physical consistency and semantic controllability. Its OCR exact-match rate reaches 91.1\%, against 44.2\% for the 12B industrial model FLUX.1 Fill [dev], and it needs only $1/14$ of that model's inference time. Leave-one-out ablations confirm that each of the three prior pathways and both loss terms contribute on their own. In downstream detection, the synthetic data raises the group-pooled AP50 of rare classes by $1.23\times$ to $7.40\times$ over a real-data-only baseline. Code and pre-trained models are available at https://github.com/52hz-whale/TrafficSignInpaint.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。