让大模型用更高效方式表达内部知识，提升特定任务表现
Self Knowledge Re-expression: A Fully Local Method for Adapting LLMs to Tasks Using Intrinsic Knowledge
- 不依赖标注数据或人类干预，直接优化模型内部知识表达方式
- 金融文档任务中信息检索召回率提升超40%，异常检测性能增33%
- 适合需要快速适应新任务且无标注数据的场景
尽管下一词预测（NTP）范式使大语言模型（LLMs）能够表达其内在知识，但其序列特性限制了在特定非生成任务上的表现。我们指出性能瓶颈源于知识表达机制，而非知识获取不足。为此，提出自知识重表达（SKR），一种无需外部监督、无需模型蒸馏的完全本地化适配方法。SKR将模型输出从通用文本生成转变为高效的任务特异性表达。在大规模金融文档数据集上的实验显示，信息检索任务的Recall@1提升超过40%，物体检测延迟降低76%以上，异常检测AUPRC提升33%以上。在MMDocRAG数据集上，性能超越领先检索模型至少12.6%。
原文摘要 · Abstract (English)
While the next-token prediction (NTP) paradigm enables large language models (LLMs) to express their intrinsic knowledge, its sequential nature constrains performance on specialized, non-generative tasks. We attribute this performance bottleneck to the LLMs' knowledge expression mechanism, rather than to deficiencies in knowledge acquisition. To address this, we propose Self-Knowledge Re-expression (SKR), a novel, task-agnostic adaptation method. SKR transforms the LLM's output from generic token generation to highly efficient, task-specific expression. SKR is a fully local method that uses only unannotated data, requiring neither human supervision nor model distillation. Experiments on a large financial document dataset demonstrate substantial improvements: over 40% in Recall@1 for information retrieval tasks, over 76% reduction in object detection latency, and over 33% increase in anomaly detection AUPRC. Our results on the MMDocRAG dataset surpass those of leading retrieval models by at least 12.6%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。