让AI理解图像和文本中的道德含义，提升对人类价值观的感知能力。
MoralCLIP: Contrastive Alignment of Vision-and-Language Representations with Moral Foundations Theory
- 基于道德基础理论，将视觉与语言中的道德线索融合到统一嵌入空间。
- 构建1.5万条带道德标注的图文数据对，实现跨模态道德对齐。
- 为开发具备道德意识的AI系统提供新方法，适合伦理与AI交叉研究者。
近年来，视觉语言模型在多模态语义理解方面取得进展，但缺乏对内容道德维度的解释与推理能力——这是人类认知的关键部分。本文提出MoralCLIP，一种基于道德基础理论（Moral Foundations Theory, MFT）的新型嵌入表示方法，通过在统一嵌入空间中整合视觉与文本的道德线索，实现跨模态道德对齐。该方法基于多标签数据集Social-Moral Image Database，识别图像内容中的共现道德基础。训练时，设计了道德数据增强策略，将标注数据扩展至1.5万张图像-文本对，每对均以MFT维度标注。实验表明，显式道德监督显著提升了单模态与多模态对道德内容的理解能力，为构建能够识别并匹配人类道德价值观的智能系统奠定基础。
原文摘要 · Abstract (English)
Recent advances in vision-language models have enabled rich semantic understanding across modalities. However, these encoding methods lack the ability to interpret or reason about the moral dimensions of content-a crucial aspect of human cognition. In this paper, we address this gap by introducing MoralCLIP, a novel embedding representation method that extends multimodal learning with explicit moral grounding based on Moral Foundations Theory (MFT). Our approach integrates visual and textual moral cues into a unified embedding space, enabling cross-modal moral alignment. MoralCLIP is grounded on the multi-label dataset Social-Moral Image Database to identify co-occurring moral foundations in visual content. For MoralCLIP training, we design a moral data augmentation strategy to scale our annotated dataset to 15,000 image-text pairs labeled with MFT-aligned dimensions. Our results demonstrate that explicit moral supervision improves both unimodal and multimodal understanding of moral content, establishing a foundation for morally-aware AI systems capable of recognizing and aligning with human moral values.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。