arXiv:2606.00829cs.CV2026-06被引 1

不训练模型,仅通过分域推理策略提升视觉语言模型在罕见场景下的问答能力。

The Right Inference Strategy Is All You Need: Nearly Training-Free Domain-Wise Inference for EgoCross Challenge

  • 针对不同领域设计专属输入、提示和答案映射策略,增强模型对关键线索的感知。
  • 在仅20样本微调的情况下,整体准确率达66.98%,接近模型潜力极限。
  • 适合资源受限场景,尤其适用于医疗、工业等小样本高难度任务部署。

EgoCross评估多模态大语言模型在显著领域偏移下的第一人称视频问答能力,测试视频来自手术、工业装配、极限运动及动物佩戴摄像头等非日常场景。在源受限赛道中,基础模型固定为Qwen3-VL-4B,官方任务特定支持集仅含20个训练样本。此设定下挑战不在于模型规模,而在于如何有效引导有限模型识别视觉、时间与答案选择的关键线索。我们发现冻结的基础模型并非无法应对这些罕见场景,而是缺乏适配新任务格式的接口。因此提出分域推理策略,分别针对四类目标领域设计不同的输入、提示与答案映射方式,使模型更易理解各类稀有场景中的关键信息。系统近乎无训练:手术与动物类问题直接使用Qwen3-VL-4B;XSports与工业类仅用官方提供的20样本进行两轮微调。最终评测中,该策略达到66.98%的整体准确率,表明精心设计的领域感知推理可弥补基础模型能力不足,恢复其原有潜力。

原文摘要 · Abstract (English)

EgoCross evaluates multimodal large language models on egocentric video question answering under substantial domain shift, where test videos come from surgery, industrial assembly, extreme sports, and animal-mounted cameras rather than ordinary daily-life scenes. In the source-limited track, the base model is fixed to Qwen3-VL-4B, while the official task-specific support set contains only 20 training samples. This setting makes the challenge less about model scaling and more about exposing the right visual, temporal, and answer-selection cues to a constrained model. Our key observation is that the frozen baseline model is not simply incapable of these rare scenarios; rather, it often fails to transfer its existing visual-language knowledge to the new task format without an appropriate interface. We therefore use a domain-wise inference strategy that treats the four target domains separately and designs different input, prompting, and answer-mapping procedures according to each domain's task characteristics. These strategies make the rare egocentric scenes more interpretable to the VLM by emphasizing the cues that matter for each domain. The resulting system is nearly training-free: surgery, and animal questions are answered with the base Qwen3-VL-4B model, while XSports and industry use only the official SFT checkpoint trained for two epochs on the provided 20 training samples. On the final evaluation, this simple strategy reaches 66.98\% overall accuracy, suggesting that careful domain-aware inference can compensate for limited base-model strength and recover much of the ability already present in the baseline model.

多模态小样本推理优化领域自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。