测试大模型对港陆影视广告语风格的识别能力，发现其依赖表面语言而非深层文化结构。
Probing Cultural Awareness in LLMs: A Case Study of Cross-Culture Aesthetic Stylistics
- 构建港台与内地影视广告语风格数据集C4STYLI进行评测
- 大模型在风格识别上表现差于人类，且不同文本领域差异显著
- 模型识别风格主要靠表层词汇，缺乏对香港特有风格结构的理解
大型语言模型（LLMs）在多元文化场景中应用日益广泛，但其对审美风格——即通过语言策略唤起文化共鸣的能力——仍缺乏深入研究。我们构建了C4STYLI基准数据集，包含来自香港和中国大陆的高风格化电影片名及广告语翻译，通过行为识别与生成能力评估大模型。大量实验表明，大模型在风格识别上表现不及人类，且跨文本领域存在差异。此外，风格识别与生成能力并不一致。为进一步检验大模型是否真正捕捉到风格信息，我们采用逻辑回归探针进行结构消融分析。结果发现，在香港语境下，大模型的风格识别主要依赖表面语言特征，而非风格结构本身，说明其对香港特有风格结构敏感度有限。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly deployed in diverse cultural contexts, yet their ability to master aesthetic stylistics, i.e., the strategic use of language to evoke cultural resonance, remains underexplored. We curate C4STYLI, a benchmark of highly stylized translated movie titles and advertising slogans from Hong Kong and the Chinese Mainland, to evaluate LLMs via the lens of behavioral recognition and productive competence. Extensive evaluations show that LLMs differ from humans in stylistic recognition, and this recognition ability varies across text domains. In addition, stylistic recognition and generation performance in LLMs are not consistently aligned. To further examine whether LLMs genuinely capture stylistic information in stylistic recognition, we conduct structural ablation with logistic regression probes. We find that, in the Hong Kong setting, stylistic recognition in LLMs relies primarily on surface-level linguistic information rather than stylistic structure. This suggests limited sensitivity to Hong Kong-specific stylistic structure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。