用专家标签提升大模型碳目标检测，自动优化提示更有效
Integrating Expert Labels into LLM-based Emission Goal Detection: Example Selection vs Automatic Prompt Design
- 用动态选例或自动优化提示整合专家标签
- 自动优化提示在769条企业报告中表现更优
- 适合需要精准提取碳目标的气候监测应用
我们研究企业报告中减排目标的检测，这是监控企业应对气候变化进展的重要任务。聚焦于如何将专家标注的文本片段融入基于大模型的流程，比较两种策略：(1) 动态选择少量示例，(2) 由大模型自动优化提示。在包含769条真实商业报告中的气候相关段落的公开数据集上，结果表明自动提示优化更具优势，而两者结合仅带来有限提升。定性分析显示，优化后的提示确实捕捉了目标提取任务的诸多细节。
原文摘要 · Abstract (English)
We address the detection of emission reduction goals in corporate reports, an important task for monitoring companies' progress in addressing climate change. Specifically, we focus on the issue of integrating expert feedback in the form of labeled example passages into LLM-based pipelines, and compare the two strategies of (1) a dynamic selection of few-shot examples and (2) the automatic optimization of the prompt by the LLM itself. Our findings on a public dataset of 769 climate-related passages from real-world business reports indicate that automatic prompt optimization is the superior approach, while combining both methods provides only limited benefit. Qualitative results indicate that optimized prompts do indeed capture many intricacies of the targeted emission goal extraction task.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。