用大模型构建中文社会文化规范库,提升对话系统的社会敏感性。
Scalable Frame-based Construction of Sociocultural NormBases for Socially-Aware Dialogues
- 以带情境框架的对话数据为输入,引导大模型生成高质量规范
- 合成数据生成的规范质量接近真实标注数据
- 适用于需要社会常识推理的对话任务,如信息检索增强
社会文化规范是社交互动中行为准则的核心,强调尊重、合作与适当行为,对对话信息检索、上下文信息检索及检索增强型机器学习等任务均有帮助。本文提出一种基于大语言模型(LLMs)的可扩展方法,构建面向社会感知对话的中文社会文化规范库(SCN Base)。该方法以包含情境框架的社会化对话作为主要数据源,通过情境约束生成过程,减少幻觉,从而提取出高质量、具细微语义的自然语言规范条目。由于真实对话中带有黄金标注框架的数据难以获取,我们采用合成数据替代。实证结果表明:(i) 由合成数据生成的规范质量与使用真实标注框架的数据相当;(ii) 使用银标签(预测)或黄金标签框架标注的真实数据所提取的规范质量显著高于无框架标注的情况。此外,我们验证了所提取的SCN在基于RAG(检索增强生成)模型中的有效性,能够支持多下游对话任务的推理。
原文摘要 · Abstract (English)
Sociocultural norms serve as guiding principles for personal conduct in social interactions, emphasizing respect, cooperation, and appropriate behavior, which is able to benefit tasks including conversational information retrieval, contextual information retrieval and retrieval-enhanced machine learning. We propose a scalable approach for constructing a Sociocultural Norm (SCN) Base using Large Language Models (LLMs) for socially aware dialogues. We construct a comprehensive and publicly accessible Chinese Sociocultural NormBase. Our approach utilizes socially aware dialogues, enriched with contextual frames, as the primary data source to constrain the generating process and reduce the hallucinations. This enables extracting of high-quality and nuanced natural-language norm statements, leveraging the pragmatic implications of utterances with respect to the situation. As real dialogue annotated with gold frames are not readily available, we propose using synthetic data. Our empirical results show: (i) the quality of the SCNs derived from synthetic data is comparable to that from real dialogues annotated with gold frames, and (ii) the quality of the SCNs extracted from real data, annotated with either silver (predicted) or gold frames, surpasses that without the frame annotations. We further show the effectiveness of the extracted SCNs in a RAG-based (Retrieval-Augmented Generation) model to reason about multiple downstream dialogue tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。