用生成模型统一处理多模态数据预测，提升跨领域准确性
Generative Distribution Prediction: A Unified Approach to Multimodal Learning
- 通过条件扩散模型生成多模态合成数据增强预测
- 在4类任务中均优于传统方法，点预测精度显著提升
- 适合需要融合文本、图像、表格数据的跨模态应用
多模态数据（包括表格、文本和视觉输入或输出）的精准预测是推动各应用领域分析进步的基础。传统方法在整合异构数据类型时往往难以保持高预测精度。我们提出生成分布预测（GDP）框架，利用多模态合成数据生成（如条件扩散模型）来提升结构化与非结构化模态的预测性能。GDP具有模型无关性，兼容任意高保真生成模型，并支持域适应的迁移学习。我们为GDP建立了严格的理论基础，证明了以扩散模型为生成核心时预测精度的统计保障。通过估计数据生成分布并适配多种损失函数进行风险最小化，GDP可在多模态场景实现精确点预测。我们在四类监督学习任务上验证了GDP：表格数据预测、问答、图像描述生成和自适应分位数回归，展示了其在不同领域的通用性与有效性。
原文摘要 · Abstract (English)
Accurate prediction with multimodal data-encompassing tabular, textual, and visual inputs or outputs-is fundamental to advancing analytics in diverse application domains. Traditional approaches often struggle to integrate heterogeneous data types while maintaining high predictive accuracy. We introduce Generative Distribution Prediction (GDP), a novel framework that leverages multimodal synthetic data generation-such as conditional diffusion models-to enhance predictive performance across structured and unstructured modalities. GDP is model-agnostic, compatible with any high-fidelity generative model, and supports transfer learning for domain adaptation. We establish a rigorous theoretical foundation for GDP, providing statistical guarantees on its predictive accuracy when using diffusion models as the generative backbone. By estimating the data-generating distribution and adapting to various loss functions for risk minimization, GDP enables accurate point predictions across multimodal settings. We empirically validate GDP on four supervised learning tasks-tabular data prediction, question answering, image captioning, and adaptive quantile regression-demonstrating its versatility and effectiveness across diverse domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。