arXiv:2505.14802cs.LG2025-05被引 1

用文本嵌入自动构建数据工程流水线,省去繁琐预处理步骤。

Text embedding models can be great data engineers

  • 将时序数据转为文本,用嵌入表示替代传统数值化处理
  • 通过变分信息瓶颈降低嵌入熵的波动,提升模型稳定性
  • 在医疗、金融等多领域超越现有基准,适合大规模自动化场景

数据工程流水线是预测分析框架中不可或缺但成本高昂的环节,需大量工程时间和领域知识完成数据摄入、预处理、特征提取与工程。本文提出ADEPT,一种基于文本嵌入的自动化数据工程框架。其核心思想是:时序数据的文本密集表示所对应的嵌入熵,可直观视为等价甚至优于传统数值密集向量表示。ADEPT采用两步法:(i) 利用文本嵌入表示多元数据源;(ii) 构建变分信息瓶颈准则,缓解时序数据文本嵌入的熵方差。该方法实现端到端预测模型自动化,即便面对缺失数据、格式错误、异常记录及不规则时间戳仍表现优异。在涵盖医疗、金融、科学及工业物联网的多类大规模数据集上,实验表明ADEPT显著优于现有最佳基准。结果证明,ADEPT有望跳过众多传统数据管道步骤,为多样化数据科学应用提供高效可扩展的自动化路径。

原文摘要 · Abstract (English)

Data engineering pipelines are essential - albeit costly - components of predictive analytics frameworks requiring significant engineering time and domain expertise for carrying out tasks such as data ingestion, preprocessing, feature extraction, and feature engineering. In this paper, we propose ADEPT, an automated data engineering pipeline via text embeddings. At the core of the ADEPT framework is a simple yet powerful idea that the entropy of embeddings corresponding to textually dense raw format representation of time series can be intuitively viewed as equivalent (or in many cases superior) to that of numerically dense vector representations obtained by data engineering pipelines. Consequently, ADEPT uses a two step approach that (i) leverages text embeddings to represent the diverse data sources, and (ii) constructs a variational information bottleneck criteria to mitigate entropy variance in text embeddings of time series data. ADEPT provides an end-to-end automated implementation of predictive models that offers superior predictive performance despite issues such as missing data, ill-formed records, improper or corrupted data formats and irregular timestamps. Through exhaustive experiments, we show that the ADEPT outperforms the best existing benchmarks in a diverse set of datasets from large-scale applications across healthcare, finance, science and industrial internet of things. Our results show that ADEPT can potentially leapfrog many conventional data pipeline steps thereby paving the way for efficient and scalable automation pathways for diverse data science applications.

数据工程文本嵌入自动化时序数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。