构建首个电子表格动作预测基准，提升自动化编辑能力。
A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets
- 基于52个真实场景的1.2万次操作序列，重建公共数据集。
- 在线评估机制支持每步预测、即时反馈与动态修正。
- 适合研究智能编辑、人机交互及低代码工具的开发者。
预测性代码补全能显著提升开发效率。然而，在更常见的电子表格中，此类自动补全功能几乎缺失。为此，我们提出一个基准，用于评估系统根据用户操作序列预测未来行为的能力。主要挑战包括：公开电子表格语料库缺乏编辑历史，以及电子表格操作空间复杂（包含空间、时间、复合操作）。为解决第一点，我们通过参数化启发式方法和大模型优化，人工构建了52条共1.2万次操作的序列，重现公共数据集。为应对第二点，我们设计了一种在线评估机制：每次用户操作后，系统生成预测，用户可接受或拒绝；若接受，则更新后续预测目标并重复该过程，直至完成目标表格。我们采用多种基线模型（包括零样本大模型、微调小模型和传统模型）进行测试，并分析了多个关键属性，如保存操作特征、误报率、效率、用户画像影响、触发条件及上下文作用。
原文摘要 · Abstract (English)
Predictive code completion greatly accelerates how quickly developers work. In spreadsheets, despite being much more common, such auto-completion features are virtually non-existent. To address this gap, we introduce a benchmark for systems that observe a sequence of user actions in a spreadsheet and predict future actions. Two challenges are (1) the absence of edit histories in public spreadsheet corpora and (2) the complex space of spreadsheet actions (spatial, temporal, composite). To address (1), we manually curate 52 sequences of 12K actions that recreate spreadsheets from public corpora, seeded by parametrized heuristics and LLM refinement. To address (2), we propose an online evaluation that expects a prediction after each user action, accepts or rejects that prediction, updates the future actions upon acceptance, and repeats this until the target spreadsheet is obtained. We use multiple baseline predictors (including zero-shot LLMs, fine-tuned SLMs, and classical models) and analyze different properties that our benchmark teaches us, including but not limited to: properties of saved actions and false positives, efficiency, effect of user profiles, effect of triggers, and effect of context.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。