构建法语道德故事数据集,评估大模型在真实社会情境中的道德对齐能力。
Histoires Morales: A French Dataset for Assessing Moral Alignment
- 基于法语文化重构道德故事,经母语者校准确保语义与文化适配。
- 发现大模型默认对齐人类道德规范,但易受用户偏好优化影响而偏离。
- 适用于研究多语言模型的道德对齐与跨文化价值观比较。
将语言模型与人类价值观对齐至关重要,尤其当它们日益融入日常生活时。尽管已有大量针对英语和中文的研究,法语在此领域的关注仍不足,导致对大模型在法语社会情境中道德推理能力的理解存在空白。为此,我们提出 Histoires Morales,一个源自 Moral Stories 的法语文本数据集,通过翻译并由母语者精修,确保语法正确性和文化适应性,并通过道德价值标注保证与法国社会规范的一致性。该数据集涵盖多种社会场景,如小费习惯差异、关系中的诚实表达以及对动物的责任。为推动后续研究,我们还对多语言模型在法语和英语数据上的对齐情况进行了初步实验,并评估了对齐的鲁棒性。结果表明,尽管大模型通常默认与人类道德规范对齐,但其倾向可被用户偏好优化轻易引导至道德或非道德方向。
原文摘要 · Abstract (English)
Aligning language models with human values is crucial, especially as they become more integrated into everyday life. While models are often adapted to user preferences, it is equally important to ensure they align with moral norms and behaviours in real-world social situations. Despite significant progress in languages like English and Chinese, French has seen little attention in this area, leaving a gap in understanding how LLMs handle moral reasoning in this language. To address this gap, we introduce Histoires Morales, a French dataset derived from Moral Stories, created through translation and subsequently refined with the assistance of native speakers to guarantee grammatical accuracy and adaptation to the French cultural context. We also rely on annotations of the moral values within the dataset to ensure their alignment with French norms. Histoires Morales covers a wide range of social situations, including differences in tipping practices, expressions of honesty in relationships, and responsibilities toward animals. To foster future research, we also conduct preliminary experiments on the alignment of multilingual models on French and English data and the robustness of the alignment. We find that while LLMs are generally aligned with human moral norms by default, they can be easily influenced with user-preference optimization for both moral and immoral data.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。