通过增强查询质量，让图像训练模型实现媲美视频监督的分割性能。
QueenVIS: Rethinking Image-Only Training for Video Instance Segmentation via Query Enrichment
- 训练时用两个辅助头提升查询的定位与空间结构能力，推理时丢弃不增加参数
- 在YouTube-VIS上比MinVIS最高提升6.7点AP，长序列任务提升10.3点
- 无需视频标注，适合资源有限但追求高精度实例分割的场景
视频实例分割（VIS）需在帧间检测、分割并追踪物体身份，主流方法依赖视频级监督。图像仅训练方法（如MinVIS）挑战这一假设，通过将帧视为独立图像，在推理时完成实例关联。然而，该方向仍被忽视。诊断分析发现，仅局部定位的查询在帧间漂移，导致关联不稳定。QueenVIS提出以查询为中心的框架：单帧训练中，为Mask2Former查询引入特征预测损失与中心预测损失，增强其与实例描述符的对齐和空间结构；推理时丢弃两头，零参数增加；通过无训练的查询传播与记忆库维持时序一致性。在ResNet-50骨干下，于YouTube-VIS和OVIS上，相较MinVIS提升最高达+6.7 AP和+4.8 AP，长序列集上提升+10.3 AP。最终在YouTube-VIS上达50.9 AP，媲美最新视频监督模型，且全程未使用任何视频片段训练。结果表明，强化查询的判别力与时序稳定性是未被充分探索的关键路径。
原文摘要 · Abstract (English)
Video instance segmentation (VIS) requires models to detect, segment, and track object identities across frames, and most methods enforce temporal consistency through video-level supervision. Image-only training approaches, with MinVIS as one prominent example, have challenged this assumption, reaching competitive VIS without video training by treating frames as independent images and associating instances only at inference. The field has nonetheless moved toward ever more elaborate video-trained trackers, which depend on costly identity-consistent annotations, leaving the image-only direction under-explored. A diagnostic analysis identifies object query quality as the bottleneck: queries trained only to localize objects within a frame drift apart across frames and destabilize association. QueenVIS introduces a query-centric framework for strengthening image-trained VIS. During single-frame training, we enrich Mask2Former queries with two auxiliary heads: a feature-prediction loss that aligns each query with the pooled backbone descriptor of its instance, and a center-prediction loss that injects spatial structure. Both heads are discarded at inference, adding zero parameters, and temporal identity is maintained by a training-free query-propagation and memory-bank scheme. On YouTube-VIS and OVIS with a ResNet-50 backbone, QueenVIS improves over MinVIS, up to +6.7 AP on YouTube-VIS, +4.8 AP on OVIS, and +10.3 AP on the long-sequence YouTube-VIS split. QueenVIS achieves 50.9 AP on YouTube-VIS and remains competitive with recent video-supervised state-of-the-art, without processing a single video clip during training. Our findings suggest that strengthening the discriminative power and temporal stability of object queries is an important, underexplored axis for VIS. Code and models: https://github.com/ArianKheir/QueenVIS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。