arXiv:2606.13693cs.CYcs.AI2026-06

用4模型共识发现:大模型推理未必提升企业环保评分效果

Limited Marginal Benefit of Reasoning-Heavy LLM Deployment in ESG Narrative Scoring: A 4-Model Consensus Study on Japanese Listed Firms

  • 四模型共识设计:1个强推理模型+3个弱推理模型对比
  • 推理模型与普通模型评分差异仅0.38分(5分制),超2分差仅2%
  • 强模型成本是普通模型的5.6倍,收益却微乎其微

利用大型语言模型(LLMs)自动化评估企业ESG叙事披露日益流行,但前沿高推理能力模型是否值得其高昂成本仍无实证定论。本研究针对日本10家上市公司的三类评估维度——量化目标、进展追踪机制和外部标准对齐——采用四模型共识设计,结合一个具推理能力的前沿模型与三个无推理能力的同期模型进行评估。在120组公司×维度×模型评分中,推理模型与各无推理模型之间的平均绝对偏差为0.38(5分制);仅有2%的成对比较出现两分及以上偏差,且无一超过两分。按公司成本核算,仅推理模型的支出约为三模型无推理集成方案的5.6倍,但结果差异极小。结论表明,在基于片段的ESG叙事评分任务中,高推理部署并未显著优于无推理共识方法,反而大幅增加运营成本。研究讨论了成本效益导向的ESG自动评分流程及实际问责场景中大模型部署治理的意义。该工作早期版本已发布于SSRN(Abstract ID 6683303)。

原文摘要 · Abstract (English)

Automated scoring of ESG narrative disclosures with large language models (LLMs) is gaining traction, yet whether reasoning-heavy frontier models add value commensurate with their cost remains empirically unsettled. We evaluate this question on a corpus of ten Japanese listed firms across three rubric axes -- quantitative targets, progress-tracking infrastructure, and external-standard alignment -- using a four-model consensus design that combines a reasoning-on frontier model with three reasoning-off contemporaries. Across 120 firm x axis x model scores, the pooled mean absolute deviation between the reasoning-on model and each reasoning-off counterpart is 0.38 on a 5-point scale; only 2% of pairwise comparisons reach a two-point deviation, and none exceeds two points. Per-firm cost accounting shows the reasoning-on arm alone costs roughly 5.6x as much as the three-provider reasoning-off ensemble, for outcomes that differ only within small margins. We conclude that in span-based ESG narrative scoring, reasoning-heavy deployment does not materially improve outcomes relative to reasoning-off consensus, while substantially increasing operational cost. We discuss implications for cost-effective ESG auto-scoring pipelines and LLM deployment governance in applied accountability settings. An earlier version of this work is available on SSRN (Abstract ID 6683303).

ESG评分大模型评估成本效益

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。