针对工程手册难理解问题，设计轻量模型精准提取与推理事实。
SMART SLM: Structured Memory and Reasoning Transformer, A Small Language Model for Accurate Document Assistance
- 分层处理：先提取三元组事实，再用向量记忆库存储，最后融合生成答案。
- 参数仅4551万，比GPT-2少64%，准确率高21.3%。
- 支持快速查证与新文档动态检索，减少幻觉，适合工程辅助场景。
工程手册内容冗长、格式密集，包含文本、操作步骤和设备参数列表，现有小型Transformer模型将其视为无结构的词元流，导致自信但错误的数值回答，且低效记忆独立事实。SMART（结构化记忆与推理转换器）提出一种更实用的解决方案：采用分层处理机制，包括（1）语法感知的事实提取器（Grammarian Tree LSTM），从手册句子中提取主语-关系-宾语三元组；（2）紧凑的索引记忆模块MANN，将这些三元组编码为384维向量并关联信息来源；（3）6层变压器，融合检索到的事实生成响应。整个模型仅使用4551万参数，比GPT-2（124M）少64%，比BERT（133M）少69%，准确率较GPT-2提升21.3%。支持双模式推理：已知文档通过索引快速路径（亚秒级响应），新上传文档则结合RAGs使用FAISS检索前20结果，记忆容量限制在64个槽位。实际部署中，该框架显著降低幻觉，提供更可靠的工程辅助结果。
原文摘要 · Abstract (English)
The user of Engineering Manuals (EM) finds it difficult to read EM s because they are long, have a dense format which includes written documents, step by step procedures, and standard parameter lists for engineering equipment. Off the shelf transformers, especially compact ones, treat this material as a flat stream of tokens. This approach leads to confident but incorrect numeric answers and forces the models to memorize separate facts inefficiently. SMART (Structured Memory and Reasoning Transformer) offers a different and practical solution to the above problem. SMART structures its processing by using a hierarchical approach, and is based upon three main job categories (1) A syntax-aware Fact Extractor (Grammarian) Tree LSTM which extracts facts as subject relation object relations from EM sentences (2) A compact indexed memory MANN (Memory Augmented Neural Network) that indexes these Rational Subject Relation Objects as 384 dimensional vectors that are associated with the source of the information, and (3) A 6 layer Transformer that learns to fuse the previously retrieved facts into its generated response. The entire SMART model utilizes 45.51M parameters, which is 64% less than GPT-2 (124M) and 69% less than BERT (133M), and it achieves a 21.3% higher accuracy than GPT-2, indicating that SMART fits the data better with the least amount of processing requirements. SMART employs dual modes of inference an indexed fast path for known documents (sub-second answer times) and an indexed dynamic path assisted by RAGs for new uploads (FAISS Top 20 results with memory severed at 64 slots). In real world deployment, this framework leads to more well supported results with reduced hallucinations than comparable small transformer models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。