解决视觉问答中的数据偏差问题，提升模型泛化能力。
Integrating Object Interaction Self-Attention and GAN-Based Debiasing for Visual Question Answering
- 引入对象交互自注意力捕捉图像中物体间关系。
- 用GAN生成无偏数据分布，增强模型鲁棒性。
- 在VQA-CP数据集上表现优于现有方法，适合对抗数据偏差场景。
视觉问答（VQA）要求模型理解并推理视觉内容以准确回答问题。现有模型常受训练数据偏差影响，过度依赖表面模式，难以泛化到多样问题和图像。本文提出IOG-VQA模型，结合对象交互自注意力与基于GAN的去偏机制。自注意力机制使模型能捕捉图像内物体间的复杂交互，提升视觉上下文理解；而基于GAN的去偏框架生成无偏数据分布，帮助模型学习更鲁棒、泛化的特征。通过融合视觉与文本信息，该模型有效缓解了数据集中的固有偏差。在VQA-CP v1和v2数据集上的大量实验表明，相比现有方法，本模型在处理有偏与不平衡数据分布时表现优异，凸显了同时关注对象交互与数据偏差对推进VQA任务的重要性。代码已开源：https://github.com/HubuKG/IOG-VQA。
原文摘要 · Abstract (English)
Visual Question Answering (VQA) presents a unique challenge by requiring models to understand and reason about visual content to answer questions accurately. Existing VQA models often struggle with biases introduced by the training data, leading to over-reliance on superficial patterns and inadequate generalization to diverse questions and images. This paper presents a novel model, IOG-VQA, which integrates Object Interaction Self-Attention and GAN-Based Debiasing to enhance VQA model performance. The self-attention mechanism allows our model to capture complex interactions between objects within an image, providing a more comprehensive understanding of the visual context. Meanwhile, the GAN-based debiasing framework generates unbiased data distributions, helping the model to learn more robust and generalizable features. By leveraging these two components, IOG-VQA effectively combines visual and textual information to address the inherent biases in VQA datasets. Extensive experiments on the VQA-CP v1 and VQA-CP v2 datasets demonstrate that our model shows excellent performance compared with the existing methods, particularly in handling biased and imbalanced data distributions highlighting the importance of addressing both object interactions and dataset biases in advancing VQA tasks. Our code is available at https://github.com/HubuKG/IOG-VQA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。