通过多模态数据采集，同时降低模型的随机与认知不确定性。
Reducing Aleatoric and Epistemic Uncertainty through Multi-modal Data Acquisition
- 设计可决策的数据采集框架，动态选择样本数量和数据模态。
- 实验证明增加模态数能降低随机不确定性，增加样本量能减少认知不确定性。
- 适合需要高可靠性预测的多模态场景，如医疗诊断、自动驾驶。
为生成准确可靠的预测，现代AI系统需融合文本、图像、音频、表格和时间序列等多种模态数据。多模态数据为分离不确定性带来新机遇与挑战：传统观点认为认知不确定性可通过收集更多数据降低，而随机不确定性不可消除。但在多模态场景下，该假设受到挑战。本文提出一种创新的数据采集框架，通过不确定性解耦实现可操作的采样决策，支持在样本规模和数据模态两个方向上进行优化。核心假设为：随着模态数量增加，随机不确定性下降；随着观测样本增多，认知不确定性减少。我们在两个多模态数据集上进行了概念验证，整合了主动学习、主动特征获取与不确定性量化思想。
原文摘要 · Abstract (English)
To generate accurate and reliable predictions, modern AI systems need to combine data from multiple modalities, such as text, images, audio, spreadsheets, and time series. Multi-modal data introduces new opportunities and challenges for disentangling uncertainty: it is commonly assumed in the machine learning community that epistemic uncertainty can be reduced by collecting more data, while aleatoric uncertainty is irreducible. However, this assumption is challenged in modern AI systems when information is obtained from different modalities. This paper introduces an innovative data acquisition framework where uncertainty disentanglement leads to actionable decisions, allowing sampling in two directions: sample size and data modality. The main hypothesis is that aleatoric uncertainty decreases as the number of modalities increases, while epistemic uncertainty decreases by collecting more observations. We provide proof-of-concept implementations on two multi-modal datasets to showcase our data acquisition framework, which combines ideas from active learning, active feature acquisition and uncertainty quantification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。