arXiv:2608.11537cs.CVcs.AI2026-08

用可观察的语义-图像接口实现精准分割,避免颜色漂移和边界混淆。

Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignment

论文配图:Generative Semantic Segmentation via an Observable Semantic-Image Interface and Hierarchical Generator Evidence Alignment
图 1 · 摘自论文原文
  • 构建显式概率接口,通过像素到类别色板的距离定义语义分布
  • 多层级特征对齐与残差修正,使分割结果在保持图像参考下更准确
  • 提出新评估方法C-IHD,无需额外模型即可提升错误像素识别能力

生成式语义分割将结构化预测以图像形式呈现,但直接色彩解码易产生颜色漂移和边界混淆;而基于潜在特征的解码器可能将渲染图像仅视为中间可视化。本文提出Semantic Prism,一种具有确定性推理的条件语义图像生成与优化框架。其扩散蒸馏的一步生成器输出语义RGB图像;每个像素到固定类别色板的颜色距离构成明确的概率接口。层次化生成器证据对齐空间对齐多层级生成器特征,并使用零初始化输出投影,在接口逻辑空间中预测加性残差,保留图像定义的接口作为最终分布参考。该接口与优化后的分布进一步支持上下文接口-层次不一致(C-IHD),一种无需辅助预测器或额外前向传播的固定读出机制,用于排名剩余像素错误。在500张图像的Cityscapes验证集上,Semantic Prism达到72.07% mIoU,比直接接口解码高11.39 mIoU,且期望校准误差仅为0.41%。在三组种子的容量匹配消融实验中,联合对齐的多层级证据表现更优。独立训练模型在BDD100K上达62.22% mIoU,而城市景观训练模型在源域冻结条件下迁移至含对应关系的恶劣天气数据集(ACDC)时仍达46.89% mIoU,无需目标域适应。在所有三组数据集上,C-IHD在相同分割预测下均优于最大softmax概率,显著提升像素错误排名的精确率-召回率曲线下面积;在ACDC上,AUPR从0.6580提升至0.7557。

原文摘要 · Abstract (English)

Generative semantic segmentation exposes structured predictions as images, but direct color decoding is susceptible to color drift and boundary mixing, whereas latent-feature decoders that predict a separate output distribution may relegate the rendered image to an intermediate visualization. We present Semantic Prism, a conditional semantic-image generation-and-refinement framework with deterministic inference. A diffusion-distilled one-step generator renders a semantic RGB image; per-pixel distances from the rendered colors to a fixed class-color codebook define an explicit probabilistic interface. Hierarchical Generator Evidence Alignment spatially aligns multi-level generator features and uses a zero-initialized output projection to predict an additive residual in the interface logit space, retaining the image-defined interface as the reference for the final distribution. The interface and refined distributions further enable Contextual Interface--Hierarchy Disagreement (C-IHD), a fixed readout for ranking remaining pixel errors without an auxiliary predictor or additional forward pass. On the 500-image Cityscapes validation set, Semantic Prism achieves 72.07% mean intersection over union, 11.39 mIoU points above direct-interface decoding, with 0.41% expected calibration error. Matched-capacity ablations over three seeds support the benefit of jointly aligned multi-level evidence. A separately trained model attains 62.22% mIoU on BDD100K, while the Cityscapes-trained model reaches 46.89\% mIoU under source-frozen transfer to the Adverse Conditions Dataset with Correspondences, without target-domain adaptation. Across all three datasets, C-IHD consistently improves the area under the precision--recall curve for pixel-error ranking over maximum softmax probability on the same segmentation predictions; on ACDC, it raises AUPR from 0.6580 to 0.7557.

语义分割生成模型图像接口误差检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。