零样本集成模型，无需微调即可实现图文检索与提示引导生成。
ZSE-Cap: A Zero-Shot Ensemble for Image Retrieval and Prompt-Guided Captioning
- 用CLIP、SigLIP和DINOv2三模型相似度加权融合做图像检索。
- 通过精心设计提示词，让Gemma 3模型将文章事件与图像内容关联。
- 零样本下获竞赛第4名，适合无标注数据场景的图文理解任务。
我们提出ZSE-Cap（零样本图文生成系统），在基于文章的图像检索与描述共享任务EVENTA中获得第四名。该方法无需在比赛数据上进行微调。图像检索方面，融合CLIP、SigLIP和DINOv2的相似度得分；图像描述方面，利用精心设计的提示词引导Gemma 3模型，使其能将文章中的高层事件与图像视觉内容对齐。系统在私有测试集上取得0.42002的最终得分，证明了基础模型集成与提示工程的有效性。代码已开源：https://github.com/ductai05/ZSE-Cap。
原文摘要 · Abstract (English)
We present ZSE-Cap (Zero-Shot Ensemble for Captioning), our 4th place system in Event-Enriched Image Analysis (EVENTA) shared task on article-grounded image retrieval and captioning. Our zero-shot approach requires no finetuning on the competition's data. For retrieval, we ensemble similarity scores from CLIP, SigLIP, and DINOv2. For captioning, we leverage a carefully engineered prompt to guide the Gemma 3 model, enabling it to link high-level events from the article to the visual content in the image. Our system achieved a final score of 0.42002, securing a top-4 position on the private test set, demonstrating the effectiveness of combining foundation models through ensembling and prompting. Our code is available at https://github.com/ductai05/ZSE-Cap.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。