arXiv:2601.13050cs.CL2026-01

用可解释的指纹分析模型简化德语文本的行为,提升诊断效率。

Profiling German Text Simplification with Interpretable Model-Fingerprints

  • 通过多维度特征生成模型简化文本的可解释指纹
  • 线性分类器识别模型配置准确率达71.9% F1分数
  • 适合需要精细调试提示策略与微调的开发者

大型语言模型虽能生成细腻的文本简化结果,但开发者缺乏高效、可复现的诊断工具。本文提出简化行为分析仪(Simplification Profiler),通过聚合多个简化样本生成模型的多维可解释指纹。该方法在德语等数据稀缺语言中尤为重要,因需适配多样目标群体而非单一风格。我们以元评估验证指纹描述能力:仅靠简单线性分类器即可可靠区分不同模型配置。实验表明,完整特征集实现最高71.9% F1得分,较基线提升超48个百分点。该工具可揭示提示策略差异及少量示例带来的细微变化,为构建更自适应的文本简化系统提供细粒度分析支持。

原文摘要 · Abstract (English)

While Large Language Models (LLMs) produce highly nuanced text simplifications, developers currently lack tools for a holistic, efficient, and reproducible diagnosis of their behavior. This paper introduces the Simplification Profiler, a diagnostic toolkit that generates a multidimensional, interpretable fingerprint of simplified texts. Multiple aggregated simplifications of a model result in a model's fingerprint. This novel evaluation paradigm is particularly vital for languages, where the data scarcity problem is magnified when creating flexible models for diverse target groups rather than a single, fixed simplification style. We propose that measuring a model's unique behavioral signature is more relevant in this context as an alternative to correlating metrics with human preferences. We operationalize this with a practical meta-evaluation of our fingerprints' descriptive power, which bypasses the need for large, human-rated datasets. This test measures if a simple linear classifier can reliably identify various model configurations by their created simplifications, confirming that our metrics are sensitive to a model's specific characteristics. The Profiler can distinguish high-level behavioral variations between prompting strategies and fine-grained changes from prompt engineering, including few-shot examples. Our complete feature set achieves classification F1-scores up to 71.9 %, improving upon simple baselines by over 48 percentage points. The Simplification Profiler thus offers developers a granular, actionable analysis to build more effective and truly adaptive text simplification systems.

文本简化模型诊断可解释性德语NLP

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。