用大模型自动还原表格缩写列名，提升数据可读性
Columbo: Expanding Abbreviated Column Names for Tabular Data Using Large Language Models
- 结合上下文与思维链推理，智能推断缩写含义
- 在五个真实数据集上比现有最佳方法高4%-29%准确率
- 适用于企业、科研等场景的表格数据清洗与理解
还原表格中缩写的列名（如将'esal'扩展为'employee salary'）对自然语言转SQL、表格问答和关键词搜索等下游任务至关重要，广泛存在于企业、科学领域和政府部门。本文提出三项关键贡献：首先，指出先前研究使用的合成数据存在严重局限，构建了四个来自企业与科学领域的全新真实数据集，包含实际缩写；其次，发现以往评估指标严重低估正确扩展数量，提出新的同义词感知评估方法，能更准确衡量性能；最后，开发Columbo——一个基于大模型的解决方案，融合上下文信息、规则约束、思维链推理与词元级分析。大量实验表明，Columbo在五个数据集上显著优于当前最先进的NameGuess方法，提升4%-29%。该系统已投入生产，应用于环境科学领域大型数据湖EDI。
原文摘要 · Abstract (English)
Expanding the abbreviated column names of tables, such as "esal" to "employee salary", is critical for many downstream NLP tasks for tabular data, such as NL2SQL, table QA, and keyword search. This problem arises in enterprises, domain sciences, government agencies, and more. In this paper, we make three contributions that significantly advance the state of the art. First, we show that the synthetic public data used by prior work has major limitations, and we introduce four new datasets in enterprise/science domains, with real-world abbreviations. Second, we show that accuracy measures used by prior work seriously undercount correct expansions, and we propose new synonym-aware measures that capture accuracy much more accurately. Finally, we develop Columbo, a powerful LLM-based solution that exploits context, rules, chain-of-thought reasoning, and token-level analysis. Extensive experiments show that Columbo significantly outperforms NameGuess, the current most advanced solution, by 4-29%, over five datasets. Columbo has been used in production on EDI, a major data lake for environmental sciences.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。