用可解释规则脚本实现中文隐喻识别,让模型决策过程透明可查。
Interpretable Chinese Metaphor Identification via LLM-Assisted MIPVU Rule Script Generation: A Comparative Protocol Study
- 将四种隐喻识别协议转化为可执行的确定性规则脚本,结合可控大模型调用。
- 跨协议对比显示不同方法差异巨大,最高一致性kappa达0.986,最低仅0.001。
- 所有方案可完全复现,理由正确率40%~87%,适合需要透明决策的研究者。
隐喻识别是修辞语言处理的基础任务,但多数计算方法为黑盒分类器,无法解释判断依据。这一可解释性差距在中文中尤为严重,因中文修辞传统丰富、缺乏形态线索且标注资源有限。本文提出一种基于大模型辅助的流水线,将四种隐喻识别协议——MIP/MIPVU词法分析、CMDAG概念映射标注、情感驱动检测和明喻导向识别——转化为可执行、人类可审计的规则脚本。每个协议由一系列确定性步骤与受控的大模型调用组成,每项分类决策均附带结构化理由。我们在七个中文隐喻数据集上评估,涵盖词级、句级和跨度级标注,首次完成中文隐喻识别的跨协议比较。协议内评估显示,协议A(MIP)在词级识别上达到F1=0.472;跨协议分析揭示显著分歧:协议A与协议D间的成对科恩κ仅为0.001,而协议B与协议C几乎完全一致(κ=0.986)。可解释性审计表明所有协议均实现100%确定性可复现,理由正确率介于0.40至0.87之间,可编辑性在0.80至1.00之间。错误分析指出概念领域错配和语体敏感性是主要失败模式。结果表明,协议选择是隐喻识别中变异的最主要来源,甚至超过模型层面的差异,且规则脚本架构在保持完全透明的同时实现了竞争性性能。
原文摘要 · Abstract (English)
Metaphor identification is a foundational task in figurative language processing, yet most computational approaches operate as opaque classifiers offering no insight into why an expression is judged metaphorical. This interpretability gap is especially acute for Chinese, where rich figurative traditions, absent morphological cues, and limited annotated resources compound the challenge. We present an LLM-assisted pipeline that operationalises four metaphor identification protocols--MIP/MIPVU lexical analysis, CMDAG conceptual-mapping annotation, emotion-based detection, and simile-oriented identification--as executable, human-auditable rule scripts. Each protocol is a modular chain of deterministic steps interleaved with controlled LLM calls, producing structured rationales alongside every classification decision. We evaluate on seven Chinese metaphor datasets spanning token-, sentence-, and span-level annotation, establishing the first cross-protocol comparison for Chinese metaphor identification. Within-protocol evaluation shows Protocol A (MIP) achieves an F1 of 0.472 on token-level identification, while cross-protocol analysis reveals striking divergence: pairwise Cohen's kappa between Protocols A and D is merely 0.001, whereas Protocols B and C exhibit near-perfect agreement (kappa = 0.986). An interpretability audit shows all protocols achieve 100% deterministic reproducibility, with rationale correctness from 0.40 to 0.87 and editability from 0.80 to 1.00. Error analysis identifies conceptual-domain mismatch and register sensitivity as dominant failure modes. Our results demonstrate that protocol choice is the single largest source of variation in metaphor identification, exceeding model-level variation, and that rule-script architectures achieve competitive performance while maintaining full transparency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。