提出CLIN框架，用简单指标评估波斯语短文学文本的创意性。
CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text
- 用话题新颖度、上下文词聚类和词汇多样性分别衡量创意三维度。
- 在波斯语文学文本上，评估结果与人类判断接近甚至更优。
- 比复杂大模型评分更高效，适合低资源语言创意评估。
评估大语言模型生成文本的创意性仍具挑战，因创意多维且以人类为中心。本文研究大模型在波斯语（低资源语言）短文学文本上的创意评估可靠性，考察多种评估策略与提示设计。发现大模型与人类的判断一致性在不同维度差异显著：结构化TTCT属性（原创性、流畅性、详尽性）一致性较高，而情感与吸引力等主观维度则明显较低。评估结果还受提示设计影响，但少样本提示、集成及多智能体辩论均未带来稳定提升。基于此，我们探索用简单可解释的代理指标近似创意维度。提出CLIN框架，分别使用话题感知新颖度（原创性）、上下文词聚类（流畅性）、词汇多样性（详尽性）。这些代理指标在本设置中达到或优于最强零样本大模型判官的人类对齐度，同时显著降低评估成本。
原文摘要 · Abstract (English)
Evaluating creativity in large language model (LLM) outputs remains challenging because creativity is multidimensional and human-centered. We examine how reliably LLMs evaluate short literary text in Persian, a low-resource language, across multiple evaluation strategies and prompt formulations. We find that LLM-human agreement varies substantially across dimensions: alignment is stronger for structured TTCT-derived properties such as Originality, Fluency, and Elaboration, but considerably weaker for more subjective dimensions, particularly Emotion and Attractiveness. Judgments are also sensitive to prompt formulation, while few-shot prompting, ensembling, and multi-agent debate provide no consistent improvement. Motivated by this dimension-dependent behavior, we investigate whether structured creativity dimensions can instead be approximated using simple, interpretable proxy metrics. We introduce CLIN, which evaluates three TTCT-derived dimensions separately using topic-aware novelty for Originality, contextual lexical clustering for Fluency, and lexical diversity for Elaboration. These proxies achieve human alignment comparable to or better than the strongest zero-shot LLM judge in our setting while requiring substantially lower evaluation cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。