轻量化多模态嵌入模型，适配离线边缘设备检索
Eddy-VL 1.9B: Structural Pruning and Layered Distillation for Edge-Deployable Multimodal Embedding
- 通过结构剪枝与分层知识蒸馏压缩模型
- 参数减少9.5%，性能保留91.7%，延迟降10%
- 适合离线取证等低延迟场景使用
本文介绍Eddy-VL 1.9B，一个基于Qwen3-VL-Embedding-2B的压缩多模态嵌入模型，专为离线、边缘部署的视觉语言检索设计。压缩方法包括：(i) 基于相邻层线性CKA的探针驱动结构剪枝，移除4个冗余文本解码器层（从28层减至24层）；(ii) 采用孔洞覆盖式师生映射的分层知识蒸馏，结合中层注意力图1-CKA及末层MSE和余弦损失，支持Matryoshka维度{128, 256, 512, 1024, 2048}。释放模型含1,926,188,032参数（bf16格式3.85 GB），较2.13B教师模型减少约9.5%。在MMEB-V2（78任务，VLM2Vec协议）上，整体得分63.2，为教师模型68.9的91.7%，恢复了剪枝导致12.1分损失中的6.4分（原仅剩56.8）。组合推理性能接近教师：SugarCrepe（86.1 vs. 86.4）、MR2-Bench（24.5 vs. 24.7）、ARO（59.5 vs. 60.4），Winoground组仍为短板（6.8 vs. 8.5）。深度剪枝使前向延迟降低约10%（NVIDIA DGX Spark上每图像从150.0降至136.4毫秒，使用FlashAttention-2）。本文展示其在受限边缘部署下多模态检索的有效性。模型权重与推理代码已公开于Hugging Face。
原文摘要 · Abstract (English)
In this report, we introduce Eddy-VL 1.9B, a compressed multimodal embedding model built on Qwen3-VL-Embedding-2B for offline, edge-deployable vision-language retrieval. Eddy-VL targets air-gapped forensic and investigative settings where cloud APIs are unavailable and low latency is essential. Compression combines (i) probe-driven structural pruning that removes four redundant text-decoder layers (28 to 24) ranked by adjacent-layer linear CKA, and (ii) layered knowledge distillation with hole-covering teacher-student mappings, mid-layer attention-map 1-CKA, and final-layer MSE and cosine losses with Matryoshka dimensions {128, 256, 512, 1024, 2048}. The released model contains 1,926,188,032 parameters (3.85 GB bf16), representing approximately 9.5% fewer parameters than the 2.13B teacher model. Empirical evaluations on MMEB-V2 (78 tasks, VLM2Vec protocol) show that Eddy-VL achieves an overall score of 63.2 compared with 68.9 for the teacher, retaining 91.7% of the teacher's performance while recovering 6.4 of the 12.1 points lost through pruning alone (56.8). Compositional reasoning performance remains close to the teacher on SugarCrepe (86.1 vs. 86.4), MR2-Bench (24.5 vs. 24.7), and ARO (59.5 vs. 60.4), while Winoground group performance (6.8 vs. 8.5) remains the primary limitation. Depth pruning also reduces forward latency by approximately 10% (150.0 to 136.4 ms per image on NVIDIA DGX Spark using FlashAttention-2). We present the architecture, compression methodology, training procedures, and evaluation results, demonstrating the effectiveness of Eddy-VL for multimodal retrieval under constrained edge deployment. Model weights and inference code are publicly available on Hugging Face.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。