arXiv:2603.26929cs.CV2026-03被引 1

让视频分割模型实时学习用户修正,减少重复操作。

Live Interactive Training for Video Segmentation

  • 模型在推理时动态学习用户修正,持续优化表现。
  • 在复杂场景下平均减少18%-34%的修正次数。
  • 轻量级更新仅需约0.5秒,适合交互式视觉应用。

交互式视频分割在遮挡、物体分离等挑战性场景中通常需要大量用户干预。现有先进模型如SAM2仅对当前帧进行修正,不保留反馈信息,导致用户反复操作。为此,我们提出实时交互训练(LIT)框架,使基于提示的视觉系统能在推理时在线学习用户修正。主要实现LIT-LoRA通过实时更新轻量级LoRA模块,在用户修正后快速适应,提升后续帧的分割性能。该方法在多个复杂视频分割基准上平均减少18%-34%的修正次数,每次修正仅增加约0.5秒训练开销。我们进一步验证其通用性,成功应用于其他分割模型,并扩展至基于CLIP的细粒度图像分类任务。本工作展示了实时自适应在减少冗余人工干预方面的巨大潜力。

原文摘要 · Abstract (English)

Interactive video segmentation often requires many user interventions for robust performance in challenging scenarios (e.g., occlusions, object separations, camouflage, etc.). Yet, even state-of-the-art models like SAM2 use corrections only for immediate fixes without learning from this feedback, leading to inefficient, repetitive user effort. To address this, we introduce Live Interactive Training (LIT), a novel framework for prompt-based visual systems where models also learn online from human corrections at inference time. Our primary instantiation, LIT-LoRA, implements this by continually updating a lightweight LoRA module on-the-fly. When a user provides a correction, this module is rapidly trained on that feedback, allowing the vision system to improve performance on subsequent frames of the same video. Leveraging the core principles of LIT, our LIT-LoRA implementation achieves an average 18-34% reduction in total corrections on challenging video segmentation benchmarks, with a negligible training overhead of ~0.5s per correction. We further demonstrate its generality by successfully adapting it to other segmentation models and extending it to CLIP-based fine-grained image classification. Our work highlights the promise of live adaptation to transform interactive tools and significantly reduce redundant human effort in complex visual tasks. Project: https://youngxinyu1802.github.io/projects/LIT/.

视频分割交互学习在线更新LoRA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。