扩充日本道德语料库,提升AI对日式道德判断的理解能力
Extended Japanese Commonsense Morality Dataset with Masked Token and Label Enhancement
- 用掩码与标签增强法生成新语句,扩展数据集至31184条
- 模型在复杂日本文化道德任务上F1达0.756,接近GPT-4 Turbo
- 适合研究跨文化AI伦理与日语语境下道德推理的学者
人工智能快速发展使得将道德推理融入系统变得至关重要,但现有模型与数据集常忽视地区与文化差异。为此,我们扩展了唯一公开的日本道德数据集JCommonsenseMorality(JCM),新版本eJCM通过提出的掩码标记与标签增强(MTLE)方法,将原始13,975条语句扩展至31,184条。MTLE利用大语言模型生成替代表达并重分配标签,精准保留道德判断关键信息。基于eJCM训练的模型取得0.857的F1分数,优于原JCM(0.837)、ChatGPT单样本分类(0.841)及AugGPT增强方法(0.850)。尤其在体现日本文化特色的复杂道德任务中,性能从0.681提升至0.756,接近GPT-4 Turbo的0.787。结果验证了eJCM的有效性与文化适配的重要性。
原文摘要 · Abstract (English)
Rapid advancements in artificial intelligence (AI) have made it crucial to integrate moral reasoning into AI systems. However, existing models and datasets often overlook regional and cultural differences. To address this shortcoming, we have expanded the JCommonsenseMorality (JCM) dataset, the only publicly available dataset focused on Japanese morality. The Extended JCM (eJCM) has grown from the original 13,975 sentences to 31,184 sentences using our proposed sentence expansion method called Masked Token and Label Enhancement (MTLE). MTLE selectively masks important parts of sentences related to moral judgment and replaces them with alternative expressions generated by a large language model (LLM), while re-assigning appropriate labels. The model trained using our eJCM achieved an F1 score of 0.857, higher than the scores for the original JCM (0.837), ChatGPT one-shot classification (0.841), and data augmented using AugGPT, a state-of-the-art augmentation method (0.850). Specifically, in complex moral reasoning tasks unique to Japanese culture, the model trained with eJCM showed a significant improvement in performance (increasing from 0.681 to 0.756) and achieved a performance close to that of GPT-4 Turbo (0.787). These results demonstrate the validity of the eJCM dataset and the importance of developing models and datasets that consider the cultural context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。