用智能体框架增强视频目标分割,提升准确率与鲁棒性。
AgentRVOS for MeViS-Text Track of 5th PVUW Challenge: 3rd Method

- 设计多角色智能体流程,分工协作完成目标定位与修正
- 通过判断目标存在性避免无效分割,显著减少误报
- 结合大模型语义理解与人工规则,适合复杂视频场景
本报告描述了一种以Sa2VA为核心的参考式视频目标分割(Ref-VOS)流水线,采用显式智能体角色分工。核心思路是:Sa2VA提供初始密集语义假设,而智能体循环决定是否接受、修改或优化该假设。流程始于目标存在性判断阶段:若目标不存在,则直接输出零掩码;否则,Sa2VA接收视频与指代提示,生成全视频的粗略掩码轨迹,作为语义先验而非最终答案。规划智能体解析查询,时间分区智能体识别关键片段,勘探智能体寻找锚帧,精修智能体将可靠掩码转为框与点,供SAM3传播。评议员评估候选轨迹,反思控制器修复弱假设,协作控制器整合多分支结果。最终系统实现:Sa2VA负责稠密语义理解,智能体层则完成存在性验证、时间搜索、置信度感知修正与最终掩码优化。
原文摘要 · Abstract (English)
This report describes a Ref-VOS pipeline centered on Sa2VA and organized with explicit agent roles. The key idea is that Sa2VA should provide the first dense semantic hypothesis, while an agent loop decides whether that hypothesis should be accepted, revised, or refined. The pipeline starts with a target-presence judgment stage. If the referred object does not exist in the video, the system directly outputs zero masks. Otherwise, Sa2VA receives the video and referring prompt and produces a coarse mask trajectory over the full video. This trajectory is treated as a semantic prior rather than a final answer. A planner agent decomposes the query, temporal partition agents identify informative blocks, scout agents search for anchor frames, and refinement agents convert reliable Sa2VA masks into boxes and points for SAM3 propagation. A critic scores candidate trajectories, a reflection controller repairs weak hypotheses, and a collaboration controller reconciles multiple agent branches. The result is a Ref-VOS system in which Sa2VA is responsible for dense grounded understanding, while the agent layer handles presence verification, temporal search, confidence-aware revision, and final mask refinement.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。