用大模型融合文本知识,让工业软传感器在少数据下也能精准预测。
A Text-Based Knowledge-Embedded Soft Sensing Modeling Approach for General Industrial Process Tasks Based on Large Language Model
- 基于大模型的文本知识嵌入框架,可捕捉时序与变量间语义关系。
- 两阶段微调使模型在少量数据下仍具强预测能力,跨任务泛化好。
- 适合工业过程建模中数据稀缺、需快速部署的场景。
数据驱动的软传感器(DDSS)已成为过程工业中关键性能指标预测的主流方法。然而,传统方法需要针对不同任务进行复杂且昂贵的定制化设计,且仅依赖结构化数据,难以融入额外上下文知识。此外,其表征学习能力有限,在数据稀缺时预测性能较差。为此,本文提出通用框架LLM-TKESS,利用大语言模型(LLM)的强大泛化求解能力、跨模态知识迁移能力及少样本学习特性,提升软传感建模效果。具体地,提出辅助变量序列编码器(AVS Encoder),挖掘序列内时序关系及辅助变量间的空间语义关联;采用两阶段微调对齐策略:第一阶段通过自回归训练与参数高效微调,快速适配过程变量数据,构建软传感基础模型(SSFM);第二阶段通过训练适配器,将SSFM适配至多种下游任务,无需修改模型结构。进一步提出两种基于文本知识嵌入的软传感器,引入自然语言模态以突破纯结构化数据的局限。得益于LLM预存的世界知识,该模型在小样本条件下表现优异。以空气预热器转子热变形为例,大量实验验证了LLM-TKESS的卓越性能。
原文摘要 · Abstract (English)
Data-driven soft sensors (DDSS) have become mainstream methods for predicting key performance indicators in process industries. However, DDSS development requires complex and costly customized designs tailored to various tasks during the modeling process. Moreover, DDSS are constrained to a single structured data modality, limiting their ability to incorporate additional contextual knowledge. Furthermore, DDSSs' limited representation learning leads to weak predictive performance with scarce data. To address these challenges, we propose a general framework named LLM-TKESS (large language model for text-based knowledge-embedded soft sensing), harnessing the powerful general problem-solving capabilities, cross-modal knowledge transfer abilities, and few-shot capabilities of LLM for enhanced soft sensing modeling. Specifically, an auxiliary variable series encoder (AVS Encoder) is proposed to unleash LLM's potential for capturing temporal relationships within series and spatial semantic relationships among auxiliary variables. Then, we propose a two-stage fine-tuning alignment strategy: in the first stage, employing parameter-efficient fine-tuning through autoregressive training adjusts LLM to rapidly accommodate process variable data, resulting in a soft sensing foundation model (SSFM). Subsequently, by training adapters, we adapt the SSFM to various downstream tasks without modifying its architecture. Then, we propose two text-based knowledge-embedded soft sensors, integrating new natural language modalities to overcome the limitations of pure structured data models. Furthermore, benefiting from LLM's pre-existing world knowledge, our model demonstrates outstanding predictive capabilities in small sample conditions. Using the thermal deformation of air preheater rotor as a case study, we validate through extensive experiments that LLM-TKESS exhibits outstanding performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。