用小模型复现大模型的长视频理解能力，仅用1.1%参数达90%精度。
Ambient @ EgoLongQA 2026: Distilling Long-Video perception into a Sub-2B Model
- 将大模型的感知模块蒸馏成20亿参数的小模型，只保留正确回答的训练轨迹。
- 在十分钟后视角视频上取得82.79分（测试集），达大模型90%性能。
- 适合资源受限场景下的长视频问答，如可穿戴设备智能推理。
本文介绍我们参与ECCV 2026可穿戴AI挑战赛中景深长视频问答赛道的参赛作品，在≤20亿参数组别中排名第一，测试集得分为0.8279。系统为单个20亿参数视觉-语言模型，可在一次贪婪前向传播中回答关于十分钟第一人称视频的多选题。该模型通过将一个工具使用型智能体流水线中的初级感知模块（而非整个智能体）蒸馏至小型学生模型实现，教师轨迹仅保留正确回答样本。最终模型仅使用原大模型1.1%的参数，达到其89%的准确率。原始基础模型在自建测试集上从27.1%提升至81.4%。该20亿参数主干模型实际含22.132亿参数，超出分组限制；为符合要求，我们将多语言嵌入表从248,320行裁剪至143,469行，使参数降至19.985亿，且被保留行的输出逻辑完全一致。
原文摘要 · Abstract (English)
We describe our entry to the EgoLongQA track of the Wearable-AI Challenge in ECCV 2026, which placed first in the <=2B parameter division with 0.8279 on the held-out test set. Our system is a single 2B vision-language model that answers multiple-choice questions about ten-minute egocentric videos in one greedy forward pass; It is obtained by distilling the junior perception module of a tool-using agentic pipeline, not the agent itself into a small student, using teacher traces filtered to those that answered correctly. it reaches 89% of the accuracy of the large agentic pipeline using 1.1% of its parameters. This raises a 27.1% base model to 81.4% on our held-out questions. The 2B backbone has 2.2132B parameters and therefore over the divisional limit, to make the entry admissable we prune the multilingual embedding table from 248,320 to 143,469 rows, reaching 1.9985B with provably identical logits on retained rows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。