七模型协作+智能裁判，生成更忠实的多轮对话。
RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generation
- 用七个不同模型加两种提示策略组成混合团队，由GPT-4o-mini挑出最优答案。
- 在26支队伍中排名第一，综合得分0.7827，远超最强基线模型（0.6390）。
- 适合关注多轮生成质量与高效模型部署的研究者和开发者。
我们提交了SemEval-2026 Task 8任务B（带参考段落生成）的获胜系统。该方法为一个包含七个大语言模型的异构集成，搭配两种提示策略，由GPT-4o-mini作为裁判每实例选出最佳候选。我们在26支参赛队中排名第一，获得0.7827的条件调和平均分，显著优于最强基线gpt-oss-120b（0.6390）。消融实验表明，模型家族、规模和提示策略的多样性至关重要，集成始终优于任一单个模型。我们还推出了Meno-Lite-0.1，一个70亿参数的领域适配模型，具备出色的性价比。同时对MTRAGEval数据集进行了分析，指出了标注局限性并提出改进方向。代码已开源：https://github.com/RaguTeam/ragu_mtrag_semeval。
原文摘要 · Abstract (English)
We present our winning system for Task~B (generation with reference passages) in SemEval-2026 Task~8: MTRAGEval. Our method is a heterogeneous ensemble of seven LLMs with two prompting variants, where a GPT-4o-mini judge selects the best candidate per instance. We ranked 1st out of 26 teams, achieving a conditioned harmonic mean of 0.7827 and outperforming the strongest baseline (gpt-oss-120b, 0.6390). Ablations show that diversity in model families, scales, and prompting strategies is essential, with the ensemble consistently beating any single model. We also introduce Meno-Lite-0.1, a 7B domain-adapted model with a strong cost--performance trade-off, and analyse MTRAGEval, highlighting annotation limitations and directions for improvement. Our code is publicly available: https://github.com/RaguTeam/ragu_mtrag_semeval
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。