arXiv:2410.00659cs.ROcs.AI2024-10被引 2

让机器人失败原因的多模态解释更连贯,避免信息矛盾。

Multimodal Coherent Explanation Generation of Robot Failures

  • 通过逻辑一致性评估,确保文本与视觉解释不冲突。
  • 用预训练文本蕴含模型微调后,准确识别多模态解释的逻辑关系。
  • 适合关注机器人可解释性、人机协作的研究者和开发者。

机器人的行为可解释性对其在社会空间中的接受度至关重要。解释机器人为何无法完成任务,对非专家用户理解其能力与局限尤为重要。目前关于机器人故障解释的研究仅限于生成文本说明,尽管已有研究证明多模态解释更具优势。然而,简单拼接多种模态可能导致不同模态间语义不连贯——这一问题尚未被充分研究。不连贯的多模态解释难以理解,甚至可能与机器人和人类的实际观察及推理过程不一致,从而导致对机器人能力的误判。本文提出一种生成连贯多模态解释的方法:通过检查不同模态解释间的逻辑一致性,并按需进行修正。我们设计了一种分类方法用于一致性评估,判断一个解释是否逻辑上遵循另一个。实验表明,微调预训练的文本蕴含识别神经网络,在多模态解释的一致性评估中表现良好。代码与数据见:https://pradippramanick.github.io/coherent-explain/

原文摘要 · Abstract (English)

The explainability of a robot's actions is crucial to its acceptance in social spaces. Explaining why a robot fails to complete a given task is particularly important for non-expert users to be aware of the robot's capabilities and limitations. So far, research on explaining robot failures has only considered generating textual explanations, even though several studies have shown the benefits of multimodal ones. However, a simple combination of multiple modalities may lead to semantic incoherence between the information across different modalities - a problem that is not well-studied. An incoherent multimodal explanation can be difficult to understand, and it may even become inconsistent with what the robot and the human observe and how they perform reasoning with the observations. Such inconsistencies may lead to wrong conclusions about the robot's capabilities. In this paper, we introduce an approach to generate coherent multimodal explanations by checking the logical coherence of explanations from different modalities, followed by refinements as required. We propose a classification approach for coherence assessment, where we evaluate if an explanation logically follows another. Our experiments suggest that fine-tuning a neural network that was pre-trained to recognize textual entailment, performs well for coherence assessment of multimodal explanations. Code & data: https://pradippramanick.github.io/coherent-explain/.

可解释性多模态机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。