用多个模型投票提升大模型生成内容的可靠性，精准率从73%升至96%。
Probabilistic Consensus through Ensemble Validation: A Framework for LLM Reliability
- 通过多模型共识机制验证生成内容，避免单一模型偏差。
- 双模型时精准率提升至93.9%，三模型达95.6%，置信区间覆盖90%以上。
- 适合医疗、金融等高风险领域，可快速部署提升AI可信度。
大语言模型在文本生成方面取得显著进展，但在医疗、法律、金融等高风险领域仍缺乏可靠性。现有方法依赖外部知识或人工干预，难以扩展。本文提出一种新框架，将集成学习用于内容验证，通过模型间共识提升准确性。在78个需事实准确性和因果一致性的复杂案例中，使用两个模型时精度从73.1%提升至93.9%（95%置信区间：83.5%-97.9%），三个模型时达95.6%（95%置信区间：85.2%-98.8%）。统计分析显示模型间一致性高（κ > 0.76），同时保持足够独立性以捕捉错误。该框架为提升关键应用中自主AI系统的可靠性提供了明确路径。尽管当前受限于选择题格式要求和处理延迟，但仍可立即应用于高可靠需求场景。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have shown significant advances in text generation but often lack the reliability needed for autonomous deployment in high-stakes domains like healthcare, law, and finance. Existing approaches rely on external knowledge or human oversight, limiting scalability. We introduce a novel framework that repurposes ensemble methods for content validation through model consensus. In tests across 78 complex cases requiring factual accuracy and causal consistency, our framework improved precision from 73.1% to 93.9% with two models (95% CI: 83.5%-97.9%) and to 95.6% with three models (95% CI: 85.2%-98.8%). Statistical analysis indicates strong inter-model agreement ($κ$ > 0.76) while preserving sufficient independence to catch errors through disagreement. We outline a clear pathway to further enhance precision with additional validators and refinements. Although the current approach is constrained by multiple-choice format requirements and processing latency, it offers immediate value for enabling reliable autonomous AI systems in critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。