用语音描述视频中运动物体，自动分割并判断目标是否存在。
Agreement-Based Audio-Visual Segmentation:Champion Report for the MeViS-Audio Track in the 8th LSVOS Challenge
- 分阶段处理：先转写语音，再用多个模型生成候选分割图。
- 选共识最高的分割结果，提升准确率至J&F 0.5952。
- 适合做多模态视频理解与语音控制的场景应用。
MeViS-Audio任务要求系统根据语音描述的运动行为，在视频中分割出对应物体，并在目标不存在时输出空掩码。本文提出一种分阶段解决方案：首先使用Qwen3-ASR将语音转为文本；随后通过多个互补的视觉定位与分割模型生成若干视频掩码轨迹；不依赖单一预测，而是选择与其他候选结果平均一致性最高的轨迹作为最终输出；针对涉及方向、数量、复数等复杂查询，引入少量显式规则进行修正；最后，利用一个视频级分类器融合视觉、音视频及视频内查询得分，判断目标是否真实存在。该系统在挑战中取得J&F 0.5952、无目标准确率0.7931、有目标准确率0.9205，最终得分0.769589，排名第一。
原文摘要 · Abstract (English)
The MeViS-Audio track asks a system to segment the objects described by a spoken motion expression throughout a video and to return empty masks when the described target is absent. We present a simple staged solution. Qwen3-ASR first converts speech into text. Several video mask tracks are then produced with complementary grounding and segmentation models. Instead of trusting a single prediction, we select the track that has the highest average mask agreement with the other candidates. A small set of explicit direction, count, and plural rules corrects queries that require more than ordinary single-object tracking. Finally, a video-level classifier combines visual, audio-visual, and within-video query scores to decide whether any target is present. The submitted system obtains 0.5952 J &F, 0.7931 no-target accuracy, 0.9205 target accuracy, and a final score of 0.769589. The challenge organizers notified our team that this result ranked first in the track.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。