让掩码变压器的推理更准，不靠调参只靠训练优化。
iFAN: Inference-Aware Learning for Plain Mask Transformers
- 训练时模拟推理过程，让预测更贴近真实输出
- 在COCO等数据集上平均提升1.20 PQ、1.30 AP
- 无需增加计算量，适合部署在资源受限场景
基于查询的掩码变换器通过最终层查询预测的像素级竞争生成分割结果，但训练过程未显式优化此推理流程。我们发现两个关键偏差：最高概率掩码得分的查询未必产生最准确掩码，且最终层解码可能丢弃中间层的优质预测。为此，提出推理感知学习（iFAN），一种适用于普通掩码变换器的通用训练框架。iFAN引入修正概率-掩码排序（APMR），使查询竞争与预测掩码质量对齐，并抑制高置信度但错误的干扰项；进一步采用跨层自蒸馏（CLSD），将更强的中间层预测传递至最终层。排名与蒸馏目标仅用于训练，推理仍保持高效的最终层解码。在COCO、ADE20K和Cityscapes上的实验表明，iFAN在全景、实例和语义分割任务中均实现稳定提升，覆盖多种架构、主干网络规模和输入分辨率。整体性能平均提升1.20 PQ、1.30 AP和0.63 mIoU，额外参数、计算量和推理延迟可忽略不计。
原文摘要 · Abstract (English)
Query-based mask transformers assemble segmentation outputs through pixel-wise competition among query predictions of the final layer, yet this inference process is not explicitly optimized during training. We identify two key mismatches: the query with the highest probability-mask score does not necessarily produce the most accurate mask, and final-layer decoding may discard superior predictions from intermediate layers. To address these issues, we propose Inference-Aware Learning (iFAN), a general training framework for plain mask transformers. iFAN introduces Adjusted Probability-Mask Ranking (APMR), which aligns query competition with predicted mask quality and suppresses high-confidence but inaccurate competitors. We further employ Cross-Layer Self-Distillation (CLSD) to transfer stronger intermediate predictions to the final layer. The ranking and distillation objectives are training-only, while inference retains efficient final-layer decoding. Experiments on COCO, ADE20K, and Cityscapes demonstrate consistent improvements across panoptic, instance, and semantic segmentation, as well as across different architectures, backbone scales, and input resolutions. Overall, iFAN improves performance by an average of 1.20 PQ, 1.30 AP, and 0.63 mIoU, with negligible additional parameters, FLOPs and inference latency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。