用高斯过程优化断字规则生成,提升准确率并缩小规则体积。
The Art of Hierarchical Competing Patterns: Gaussian Process Optimization of Hyphenation
- 用高斯过程自动搜索最优断字参数配置
- 17个数据集上16个提升准确率,平均规则体积减至40.7%
- 适合需要稳定、可复现断字方案的系统开发
断字模式仍是排版系统、文本编辑器和网页渲染引擎中广泛使用的紧凑词切分方案,但其生成仍依赖人工调参。本文将patgen参数配置选择建模为黑箱超参数优化问题,并评估高斯过程贝叶斯优化在此任务上的表现。优化目标结合了面向精度的F_{1/7}-分数与显式的前缀树大小-准确率权衡,通过归一化前缀树大小惩罚项实现。在覆盖14种语言和多种文字的17个断字数据集上进行评估。相较于从相同8/10训练集重新生成的两个强基线手调参数配置,在相同的1/10测试集上,GP优化配置在16个数据集上提升F_{1/7},所有17个数据集均减少前缀树大小,中位数优化/基线前缀树大小比为0.407。数据集级符号检验p值为1.37e-4;在五个代表性数据集上进行预算匹配对比显示,系统性搜索表现不逊于甚至优于最佳手调配置。结果表明,基于模型的优化能提升断字模式生成的可复现性,降低对专家试错的依赖,同时保持准确率与紧凑性的显式权衡。
原文摘要 · Abstract (English)
Hyphenation patterns remain a compact and widely deployed solution for word breaking in typesetting systems, text processors, and web rendering engines, but their generation still depends on manually tuned patgen program parameter profiles. We formulate patgen profile selection as a black-box hyperparameter optimization problem and evaluate Gaussian-process Bayesian optimization for this task. The search objective combines a precision-oriented F_{1/7}-score with an explicit trie size-accuracy trade-off using a normalized trie-size penalty. We evaluate the method on 17 hyphenated word-list datasets covering 14 languages and multiple scripts. Against two strong hand-tuned profiles regenerated from the same 8/10 training split and evaluated on the same 1/10 held-out test split, the GP-optimized profiles improve F_{1/7} on 16 of 17 datasets and reduce trie size on all 17. The median optimized/baseline trie ratio is 0.407. A dataset-level sign test gives p = 1.37e-4; a separate budget-matched comparison on five representative datasets shows that systematic search is competitive and usually improves over the best hand-tuned profile under the fixed comparison objective. The results show that model-based optimization can make pattern generation more reproducible and less dependent on expert trial-and-error while keeping the accuracy-compactness trade-off explicit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。