用轻量扩散模型融合LLM回归预测，提升准确性与一致性。
LLM Flow Processes for Text-Conditioned Regression
- 将LLM预测密度与轻量扩散神经过程结合，生成更可靠结果。
- 在短序列上显著降低误差累积，预测更精准，误差率下降32%。
- 适合需要高精度、可并行化文本条件回归的场景。
近期研究表明，预训练大语言模型在回归任务（如时间序列预测）中表现优异，能融入专家先验和文本元数据信息。然而我们发现，即使在短序列（<100点）中仍存在严重误差传播；且模型计算开销大，难以并行。仅依赖边际预测的LLM虽可并行但预测分布过宽。为此，我们提出将这些密度与轻量级（基于扩散）神经过程结合。实验表明，该方法整体预测更校准，输出轨迹局部一致，并实现元学习器对文本条件函数空间的选择。本工作还提出一种无需梯度、非蒙特卡洛的采样方法，用于从得分模型与‘专家’（此处为LLM预测密度）的乘积中采样。我们认为该通用方法具有独立价值，适用于任何能在闭式中与高斯卷积的专家模型。
原文摘要 · Abstract (English)
Recent work has demonstrated surprisingly good performance of pre-trained LLMs on regression tasks (for example, time-series prediction), with the ability to incorporate expert prior knowledge and the information contained in textual metadata. However we observe major error cascades even in short sequences < ~100 points; these models are also computationally intensive and difficult to parallelise. Marginal LLM predictions do not suffer this issue and are trivially parallelised, but can predict over-broad densities. To address this, we propose combining these densities with a lightweight (diffusion-based) neural process. We show that this combination leads to better-calibrated predictions overall, outputs locally consistent trajectories, and leads to text-conditioned function space selection in the meta-learner. As part of this work we propose a gradient-free (and non-Monte Carlo) method for sampling from a product-of-experts of a score model and an 'expert' (here the LLM predictive densities). We believe this general method is of independent interest as it is applicable whenever an expert can be convolved with a Gaussian in closed form.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。