直接嵌入时间步并对比对齐,提升时序问答模型表现
Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering
- 用线性编码器直接将时间步映射到语言模型空间,避免分块与分词损失
- 在六个任务上超越开源和商用大模型基线,性能全面提升
- 适合需要精确时间信息的时序分析场景,如金融、医疗预测
大语言模型的发展催生了时序问答(TSQA),将时序分析转化为自然语言问答。然而,直接输入原始数值序列存在分词瓶颈:字节对编码会将连续值拆分为不稳定的标记,丢失数值大小、量级和趋势信息。现有方法采用固定窗口的分块编码器,锁定单一粒度,破坏模式且隐藏确切时间步,还需额外模块,难以跨数据集迁移。为此,我们提出CADE(对比对齐与直接嵌入)框架,包含两大核心:直接时间步嵌入与语义对齐。该框架通过点对点线性编码器和MLP投影器,将每个时间步直接映射至语言模型嵌入空间,保留索引级访问能力,无需分块与填充。为进一步弥合时序与语言表征间的语义差距,引入单向监督对比损失,对齐时序嵌入与冻结的类别名文本锚点。在公开的Time-MQA基准上的实验表明,该框架在六项TSQA任务中持续优于开源与专有大模型基线。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have given rise to time-series question answering (TSQA), which formulates time-series analysis as natural-language question answering. However, directly feeding raw numerical series into LLMs suffers from a tokenization bottleneck: Byte Pair Encoding fragments continuous values into unstable tokens whose embeddings lack meaningful metric structure, resulting in the loss of magnitude, scale, and trend information. Prior methods use patch-based encoders that split the series into fixed windows, locking in one granularity that breaks patterns and hides exact timesteps, through a separate module that rarely transfers across datasets with different lengths or sampling rates. To address this challenge, we propose CADE (Contrastive Alignment with Direct Embedding), a novel framework for TSQA built upon two key components: direct timestep embedding and semantic alignment. The proposed framework maps each timestep directly into the LLM embedding space through a point-wise linear encoder and MLP projector, preserving exact index-level access while eliminating the need for patching and padding. To further bridge the semantic gap between time-series and language representations, we introduce a novel one-directional supervised contrastive loss that aligns time-series embeddings with frozen class-name text anchors. Experimental results on the public Time-MQA benchmark demonstrate that our framework consistently improves performance across six TSQA tasks, outperforming both open-source and proprietary LLM baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。