arXiv:2505.11604cs.CL2025-05ACL被引 7

用自然语言直接操作幻灯片结构数据,比传统视觉方法更快更准

Talk to Your Slides: High-Efficiency Slide Editing via Language-Driven Structured Data Manipulation

  • 通过解析幻灯片底层对象模型实现精准编辑,不依赖图像识别
  • 文本类任务处理速度提升34%,成本降低87%,指令理解准确率高34%
  • 适合需要批量修改、格式统一的职场人士和内容创作者

幻灯片编辑是常见但繁琐的任务,涵盖从创意排版到重复文本维护。现有基于多模态大模型的图形界面代理在需视觉感知的任务(如布局调整)上表现良好,但在处理以文本为中心或批量操作时计算开销大、延迟高。本文提出Talk-to-Your-Slides,一种通过自然语言驱动结构化数据操作的高效幻灯片编辑代理,绕过图像模态,直接操作幻灯片底层对象模型。该方法确保内容修改精确且风格一致,克服了依赖OCR视觉代理的局限。系统采用分层架构,将高层指令精准映射至低层执行代码。实验表明,在文本与格式任务中,本方法处理速度提升34%,指令契合度提高34%,运行成本降低87%。此外,我们构建了TSBench基准数据集,包含379条经人工验证的指令,其中包含专为评估复杂视觉依赖查询鲁棒性的Hard子集。代码与数据集已开源。

原文摘要 · Abstract (English)

Editing presentation slides is a frequent yet tedious task, ranging from creative layout design to repetitive text maintenance. While recent GUI-based agents powered by Multimodal LLMs (MLLMs) excel at tasks requiring visual perception, such as spatial layout adjustments, they often incur high computational costs and latency when handling structured, text-centric, or batch processing tasks. In this paper, we propose Talk-to-Your-Slides, a high-efficiency slide editing agent that operates via language-driven structured data manipulation rather than relying on the image modality. By leveraging the underlying object model instead of screen pixels, our approach ensures precise content modification while preserving style fidelity, addressing the limitations of OCR-based visual agents. Our system features a hierarchical architecture that effectively bridges high-level user instructions with low-level execution codes. Experiments demonstrate that for text-centric and formatting tasks, our method enables 34% faster processing, achieves 34% better instruction fidelity, and operates at an 87% lower cost compared to GUI-based baselines. Furthermore, we introduce TSBench, a human-verified benchmark dataset comprising 379 instructions, including a Hard subset designed to evaluate robustness against complex and visually dependent queries. Our code and benchmark are available at https://github.com/KyuDan1/Talk-to-Your-Slides.

幻灯片编辑自然语言控制效率优化结构化数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。