用更简单的整体式提示，能和复杂分解式提示一样甚至更好判断答案是否完整支持。
Rethinking Atomic Decomposition for LLM Judges: A Prompt-Controlled Study of Reference-Grounded QA Evaluation
- 用整体式提示直接判断答案，避免拆解再验证的复杂流程。
- 在两个基准上整体式提示表现优于分解式，尤其擅长发现不完整回答。
- 适合需要高效、准确评估答案完整性的评测场景。
原子分解——将候选答案拆分为多个主张并逐一与参考文本核对——是大模型参考依据型评判器的常见设计。然而，原子提示通常更长更丰富，难以区分优势来自分解本身还是提示更详细。本文研究了基准式完整性敏感的参考支撑分类任务：判断候选答案相对于参考文本是否完全支持、部分支持或不支持。对比自分解原子评判器（单提示拆解验证）与相同输入、类似详细评分标准的提示控制整体评判器。在TruthfulQA、ASQA、QAMPARI三个数据集各200个源例上，使用四种模型族、源级配对测试、聚类自助法及每种设计家族三个预冻结提示变体的聚合分析，结果表明整体评判器在两个数据集上表现匹配或超越原子评判器：在ASQA和QAMPARI上，所有四个模型族均偏好整体式（三族统计显著）；而TruthfulQA显示微弱原子优势。整体式优势集中在部分支持案例——即不完整性检测。敏感性分析与人工标注对比确认，该排名在基准完整性与人类事实正确性标准下均成立。该发现仅针对三种问答型基准上自分解单提示模式，多阶段原子流水线与非问答任务仍待验证。在考察的扰动中，参考质量下降对两类评判器的影响最大，导致精度显著下降。
原文摘要 · Abstract (English)
Atomic decomposition -- breaking a candidate answer into claims before verifying each against a reference -- is a widely adopted design for LLM-based reference-grounded judges. However, atomic prompts are typically richer and longer, making it unclear whether any advantage comes from decomposition or from richer prompting. We study this for benchmark-style completeness-sensitive reference-support classification: classifying a candidate as fully supported, partially supported, or unsupported relative to a supplied reference. We compare a self-decomposing atomic judge (single-prompt decompose-and-verify) against a prompt-controlled holistic judge with the same inputs and a similarly detailed rubric. On 200 source examples per dataset across TruthfulQA, ASQA, and QAMPARI, with four model families, source-level paired tests, cluster bootstrap, and aggregation across three pre-frozen prompt variants per design family, we find the holistic judge matches or exceeds the atomic judge on two of three benchmarks: ASQA and QAMPARI favor holistic across all four families (statistically reliable in three of four), while TruthfulQA shows a small atomic edge. The holistic advantage is concentrated in partially\_supported cases -- incompleteness detection. A sensitivity check against human annotations confirms the ranking under both benchmark-completeness and human factual-correctness standards. Our finding is specific to the self-decomposing single-prompt pattern on three QA-style benchmarks with 200 source examples each; multi-stage atomic pipelines and non-QA tasks remain untested. Among perturbations examined, reference-quality degradation produced the largest accuracy drops for both judge families.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。