arXiv:2606.19626cs.AIcs.CL2026-06

让技术文本中的物理量保持完整,提升中文以外语言的量化推理能力

Toten: A Knowledge-Based System For Structure-Preserving Representation Of Physical Quantities And Technical Notation In Brazilian Portuguese

  • 基于工程实体本体的显式分类系统,确保物理量整体输入
  • 在巴西葡萄牙语数据集上实现90.4%的数值重建率,优于现有方法
  • 适合需要精确处理技术符号与单位的科研与工业场景

依赖定量推理的AI流程要求技术文本中物理量、数字、单位和符号表达保持完整;而字节对编码(BPE)会将这些实体切碎,尤其在技术性巴西葡萄牙语中问题更严重。本文提出TOTEN,一种基于知识的系统,以完整类型单元形式保留每项技术实体:词汇不通过统计生成,而是按工程实体本体(OEE)显式分类。核心由三元组 <O, classify, {inst_tau}> 构成:类型、原则与不变量;一个将原始文本映射为带类型区域的分类器;以及生成自描述表示的实例化器。系统通过与三个外部权威强制耦合保障完整性:Pint(量纲)、Unicode字符数据库(排版)、RSLP(葡萄牙语形态)。在内部基准EngQuant(N=800)及四个外部巴西葡萄牙语语料库(共1771个有效案例)上评估原子性、量纲等价性、排版鲁棒性与数值重建四项属性,报告检测召回率。相较八种先进基线,TOTEN在所有对比中实现单位原子性,并在外源数据上达到0.775–0.904的重建率(最佳基线为0.627–0.703),在EngQuant上为0.780(基线0.340)。差异显著(McNemar,Holm校正)。内部与外部排名的相关性验证了基准的并发效度。TOTEN在量纲等价性上与Pint无统计差异。结果是结构忠实、可审计、低成本的技术知识输入层,无需生成模型。

原文摘要 · Abstract (English)

AI pipelines that reason quantitatively over technical text depend on input where physical quantities, numbers, units, and symbolic expressions arrive intact; when these entities fragment at tokenization, errors propagate downstream. Byte-Pair Encoding, optimized for vocabulary compression, is blind to such entities and fragments them into arbitrary subwords -- a problem aggravated in technical Brazilian Portuguese. We present TOTEN, a knowledge-based system whose input representation preserves each technical entity as a whole, typed unit: vocabulary is not derived statistically but classified declaratively under a formal ontology of engineering entities (OEE). The core is the triple <O, classify, {inst_tau}>: types, principles, and invariants; a classifier mapping raw text into typed regions; and instantiators yielding a self-descriptive representation. Integrity rests on deterministic coupling to three external authorities: Pint (dimensional), Unicode Character Database (typographic), and RSLP (Portuguese morphology). We evaluate four properties verifiable by construction -- atomicity, dimensional equivalence, typographic robustness, numerical reconstruction -- on an internal benchmark (EngQuant, N=800) and four Brazilian Portuguese external corpora (N=1771 eligible cases), and report detection recall. Against eight state-of-the-art baselines, TOTEN reaches unit atomicity in all contrasts and reconstruction of 0.775-0.904 externally vs. 0.627-0.703 for the best (Quantulum3); on EngQuant, 0.780 vs. 0.340. Differences are significant (McNemar, Holm-corrected). Spearman correlation between internal and external rankings confirms concurrent validity of the control benchmark. TOTEN shows statistical parity with Pint in dimensional equivalence. The result is a structurally faithful, auditable, low-cost input layer for intelligent systems on technical knowledge, without generative models.

知识系统符号保真多语言处理技术文本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。