融合视频与工艺参数，实时预测饼干干燥完成时间
Multi-Modal Fusion of In-Situ Video Data and Process Parameters for Online Forecasting of Cookie Drying Readiness
- 用双编码器+注意力解码器融合视频与工艺数据
- 预测误差仅15秒，比顶尖方法快65.69%
- 适合工业场景的轻量高效在线决策系统
食品干燥对生产、保质期和运输成本至关重要。准确的实时干燥完成预测有助于降低能耗、提升效率并保证品质。但因干燥过程动态性强、数据有限且缺乏有效分析方法，实现难度大。为此，我们提出一种端到端多模态数据融合框架，将现场视频数据与工艺参数结合，用于实时预测饼干干燥完成时间。模型采用新型编码器-解码器结构，包含模态专用编码器与基于Transformer的解码器，能有效提取特征并保留各模态独特结构。实验表明，该模型平均预测误差仅为15秒，在糖饼干干燥任务中，相比现有最优数据融合方法提升65.69%，较纯视频模型提升11.30%。同时兼顾精度、模型大小与计算效率，适用于异构工业数据集。该方法可扩展至多种工业多模态融合任务，支持在线决策。
原文摘要 · Abstract (English)
Food drying is essential for food production, extending shelf life, and reducing transportation costs. Accurate real-time forecasting of drying readiness is crucial for minimizing energy consumption, improving productivity, and ensuring product quality. However, this remains challenging due to the dynamic nature of drying, limited data availability, and the lack of effective predictive analytical methods. To address this gap, we propose an end-to-end multi-modal data fusion framework that integrates in-situ video data with process parameters for real-time food drying readiness forecasting. Our approach leverages a new encoder-decoder architecture with modality-specific encoders and a transformer-based decoder to effectively extract features while preserving the unique structure of each modality. We apply our approach to sugar cookie drying, where time-to-ready is predicted at each timestamp. Experimental results demonstrate that our model achieves an average prediction error of only 15 seconds, outperforming state-of-the-art data fusion methods by 65.69% and a video-only model by 11.30%. Additionally, our model balances prediction accuracy, model size, and computational efficiency, making it well-suited for heterogenous industrial datasets. The proposed model is extensible to various other industrial modality fusion tasks for online decision-making.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。