用自动化方法优化动态规划算法,提升自然语言处理效率
Automating the Analysis and Improvement of Dynamic Programming Algorithms with Applications to Natural Language Processing
- 用领域专用语言 Dyna 描述动态规划程序
- 自动搜索变换序列,将运行时间复杂度显著降低
- 适合算法优化与NLP研究者快速改进现有代码
本论文提出一个自动化系统,用于分析和改进动态规划算法,这类算法在自然语言处理和计算机科学中已应用数十年。编写正确且具有最优渐近时间复杂度的程序往往难以直观完成,耗时且易出错。本文通过五部分设计实现自动化:1)使用领域专用语言 Dyna 精炼描述动态规划程序;2)通用求解器高效执行程序;3)静态分析系统提供类型及最坏情况时间/空间复杂度分析;4)一套保持语义的程序变换集合,归纳了文献中众多作者加速算法的重复洞察;5)搜索算法自动寻找一系列变换以降低初始正确程序的运行复杂度。实验证明,该系统可如人类程序员般发现显著性能提升,许多文献中报道的速度优化均可被自动发现。原型系统已开源,地址为 https://github.com/timvieira/dyna-pi。
原文摘要 · Abstract (English)
This thesis develops a system for automatically analyzing and improving dynamic programs, such as those that have driven progress in natural language processing and computer science, more generally, for decades. Finding a correct program with the optimal asymptotic runtime can be unintuitive, time-consuming, and error-prone. This thesis aims to automate this laborious process. To this end, we develop an approach based on 1. a high-level, domain-specific language called Dyna for concisely specifying dynamic programs 2. a general-purpose solver to efficiently execute these programs 3. a static analysis system that provides type analysis and worst-case time/space complexity analyses 4. a rich collection of meaning-preserving transformations to programs, which systematizes the repeated insights of numerous authors when speeding up algorithms in the literature 5. a search algorithm for identifying a good sequence of transformations that reduce the runtime complexity, given an initial, correct program We show that, in practice, automated search -- like the mental search performed by human programmers -- can find substantial improvements to the initial program. Empirically, we show that many speed-ups described in the NLP literature could have been discovered automatically by our system. We provide a freely available prototype system at https://github.com/timvieira/dyna-pi.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。