arXiv:2510.19886cs.CL2025-10被引 3

首个专家评估的LLM在生命周期评价中表现基准,揭示其幻觉风险与实用价值。

An Expert-grounded benchmark of General Purpose LLMs in LCA

  • 邀请17位专家对11个主流LLM在22项LCA任务中的输出进行评估
  • 37%回答含错误信息,部分模型幻化引用率高达40%
  • 开源与闭源模型表现接近,小模型也能提供良好解释质量

目的:人工智能,尤其是大语言模型(LLMs),正被探索用于支持生命周期评估(LCA)。尽管已在环境和社会领域有示范应用,但其可靠性、鲁棒性和可用性的系统性证据仍有限。本研究提供了首个基于专家意见的LLM在LCA中的基准评估,填补了该领域缺乏标准化评估框架的空白。方法:我们评估了11个通用型LLM,涵盖商业和开源模型家族,在22个与LCA相关的任务上进行测试。17位经验丰富的从业者依据科学准确性、解释质量、鲁棒性、可验证性及指令遵循度等标准,对模型输出进行了评审,共收集168份专家反馈。结果:专家判断37%的回应包含不准确或误导性信息。多数模型在准确性与解释质量上被评为平均或良好,即使小型模型亦然;格式遵循性普遍获好评。幻觉率差异显著,某些模型生成虚构引用的比例高达40%。在开放权重与闭源模型之间未见明显优劣区分,开源模型在准确性与解释质量方面表现优于或媲美闭源模型。结论:这些发现突显了在未加约束的情况下直接使用LLM进行LCA的风险,例如将其视为自由形式的“预言机”,同时也表明其在解释质量和减轻简单任务劳动强度方面的优势。若无适当的校准机制,通用型LLM的应用存在潜在偏差。

原文摘要 · Abstract (English)

Purpose: Artificial intelligence (AI), and in particular large language models (LLMs), are increasingly being explored as tools to support life cycle assessment (LCA). While demonstrations exist across environmental and social domains, systematic evidence on their reliability, robustness, and usability remains limited. This study provides the first expert-grounded benchmark of LLMs in LCA, addressing the absence of standardized evaluation frameworks in a field where no clear ground truth or consensus protocols exist. Methods: We evaluated eleven general-purpose LLMs, spanning both commercial and open-source families, across 22 LCA-related tasks. Seventeen experienced practitioners reviewed model outputs against criteria directly relevant to LCA practice, including scientific accuracy, explanation quality, robustness, verifiability, and adherence to instructions. We collected 168 expert reviews. Results: Experts judged 37% of responses to contain inaccurate or misleading information. Ratings of accuracy and quality of explanation were generally rated average or good on many models even smaller models, and format adherence was generally rated favourably. Hallucination rates varied significantly, with some models producing hallucinated citations at rates of up to 40%. There was no clear-cut distinction between ratings on open-weight versus closed-weight LLMs, with open-weight models outperforming or competing on par with closed-weight models on criteria such as accuracy and quality of explanation. Conclusion: These findings highlight the risks of applying LLMs naïvely in LCA, such as when LLMs are treated as free-form oracles, while also showing benefits especially around quality of explanation and alleviating labour intensiveness of simple tasks. The use of general-purpose LLMs without grounding mechanisms presents ...

LLM评估生命周期评价专家评测幻觉检测

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。