用结构化图像表示提升机器人决策可解释性
SIR: Structured Image Representations for Explainable Robot Learning
- 以场景图作为中间表示，通过稀疏化生成任务相关子图
- 在RoboCasa上成功率达19.5%，优于基线14.81%
- 可分析模型偏差，发现数据中的虚假关联与位置偏倚
基于学习视觉嵌入的机器人策略缺乏显式结构，对视觉干扰敏感，导致行为决策过程不透明。为此，我们提出结构化图像表示（SIR），利用场景图（SGs）作为机器人策略学习的中间表示。方法首先构建全连接图，以图像特征初始化节点；随后模块端到端学习稀疏化，生成任务相关的子图并传递给动作生成模型，使模型具有内在可解释性。在RoboCasa上的评估显示，稀疏图策略平均成功率达19.5%，优于基于图像的基线方法（14.81%）。更重要的是，所学稀疏图成为模型分析的强大工具：通过分析子图与人类预期的偏离情况，如包含干扰物或遗漏关键物体，成功揭示了数据集中的偏差，包括虚假相关性和位置偏倚。
原文摘要 · Abstract (English)
Existing robot policies based on learned visual embeddings lack explicit structure and are sensitive to visual distractions. Thus, the representations that drive their behaviour are often opaque, making their decision-making process difficult to interpret. To address this, we introduce Structured Image Representations (SIR), a method that leverages Scene Graphs (SGs) as an intermediate representation for robot policy learning. Our approach first constructs a fully connected graph, using image-derived features as initial node representations. Then, a module learns to sparsify this graph end-to-end, creating a task-relevant sub-graph that is passed to the action generation model. This process makes our model intrinsically explainable. Evaluations on RoboCasa show that our sparse graph policies outperform image-based baselines on average with 19.5% vs 14.81% success rate. Most importantly, we show that the learned sparse graphs are a powerful tool for model analysis. By analysing when the model's sub-graph deviates from human expectation, such as by including distractor nodes or omitting key objects, we successfully uncover dataset biases, including spurious correlations and positional biases. https://github.com/intuitive-robots/SIR_Model
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。