通过多次调用模型并聚类评分，提升大模型回答的准确性与可靠性。
RECSIP: REpeated Clustering of Scores Improving the Precision
- 并行调用多个大模型生成答案，基于评分进行聚类分析。
- 在MMLU-Pro基准上使准确率提升5.8个百分点，超越单个最优模型。
- 适合对结果可靠性要求高的高风险场景或工业应用。
大型语言模型（LLMs）在自然语言处理领域取得了显著进展，但其随机架构导致响应可靠性不足，可能在高风险环境或工业场景中引发严重危害或高昂失败。为此，我们提出RECSIP框架——通过并行调用多个模型、对输出结果进行评分与聚类，以提升响应精度与可信度。在MMLU-Pro基准上，使用GPT-4o、Claude和Gemini的参考实现recsip，相比最优单模型，整体准确率提升5.8个百分点。
原文摘要 · Abstract (English)
The latest research on Large Language Models (LLMs) has demonstrated significant advancement in the field of Natural Language Processing (NLP). However, despite this progress, there is still a lack of reliability in these models. This is due to the stochastic architecture of LLMs, which presents a challenge for users attempting to ascertain the reliability of a model's response. These responses may cause serious harm in high-risk environments or expensive failures in industrial contexts. Therefore, we introduce the framework REpeated Clustering of Scores Improving the Precision (RECSIP) which focuses on improving the precision of LLMs by asking multiple models in parallel, scoring and clustering their responses to ensure a higher reliability on the response. The evaluation of our reference implementation recsip on the benchmark MMLU-Pro using the models GPT-4o, Claude and Gemini shows an overall increase of 5.8 per cent points compared to the best used model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。