用专业诗人评估，测出ChatGPT懂现代诗的真实水平
Can ChatGPT Really Understand Modern Chinese Poetry?
- 联合诗人从多维度评测模型对现代诗的理解
- 73%情况下能准确捕捉诗人原意，但诗意把握不足
- 为大模型诗歌理解研究提供可复现的评估框架
ChatGPT在诗歌生成与翻译上表现优异，但其对诗歌的真实理解能力仍不明晰。以往研究仅分析实验结果，未触及理解本质。本文提出一个全面评估框架，联合专业诗人对ChatGPT解读现代中文诗歌的表现进行多维评价。结果显示，73%情况下其解读与原作者意图相符，但在捕捉诗意等维度表现欠佳。该研究不仅验证了模型理解能力，也建立了未来大模型诗歌应用研究的基础。评估涵盖多位诗人作品，确保结果代表性。
原文摘要 · Abstract (English)
ChatGPT has demonstrated remarkable capabilities on both poetry generation and translation, yet its ability to truly understand poetry remains unexplored. Previous poetry-related work merely analyzed experimental outcomes without addressing fundamental issues of comprehension. This paper introduces a comprehensive framework for evaluating ChatGPT's understanding of modern poetry. We collaborated with professional poets to evaluate ChatGPT's interpretation of modern Chinese poems by different poets along multiple dimensions. Evaluation results show that ChatGPT's interpretations align with the original poets' intents in over 73% of the cases. However, its understanding in certain dimensions, particularly in capturing poeticity, proved to be less satisfactory. These findings highlight the effectiveness and necessity of our proposed framework. This study not only evaluates ChatGPT's ability to understand modern poetry but also establishes a solid foundation for future research on LLMs and their application to poetry-related tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。