通过合成文本对比分析，挖掘大模型输出中群体相关的隐性语言模式。
Contrastive Analysis of Linguistic Representations in Large Language Model Outputs through Structured Synthetic Data Generation and Abstracted N-gram Associations
- 构建情境可控的合成文本对，仅改变目标群体标识以隔离语言差异。
- 用改进的互信息量化语言抽象与群体的关联强度，识别偏差表达。
- 通过片段排序聚焦高偏差信号区域，辅助专家判断语境危害性。
我们提出一种方法论框架，通过对比合成文本生成与统计分析，发现与不同社会群体相关的语言和话语模式。与以往依赖预定义词汇列表诊断偏见的方法不同，本研究旨在刻画微妙的偏见表达，并基于上下文数据而非孤立词句进行分析。该方法适用于叙事、任务导向或对话等多种文本类型。通过控制情境场景与群体标记的组合，生成仅在所指群体上不同的最小文本对，保持叙事条件一致。为实现稳健分析，将语言形式抽象化，并使用改进的点互信息（pointwise mutual information）量化语言抽象与群体间的关联强度，以检测跨群体出现频率异常的语言表达。采用片段排序策略，优先筛选出包含高浓度偏差语言信号的文本段落，使专家能够结合语境评估这些表达的潜在危害，实现量化分析与定性解读的衔接。
原文摘要 · Abstract (English)
We present a methodological framework to discover linguistic and discursive patterns associated to different social groups through contrastive synthetic text generation and statistical analysis. In contrast with previous approaches, we aim to characterize subtle expressions of bias, instead of diagnosing bias through a pre-determined list of words or expressions. We are also working with contextualized data instead of isolated words or sentences. Our methodology applies to textual productions in any genre, encompassing narrative, task-oriented or dialogic. Contextualized data are generated using controlled combinations of situational scenarios and group markers, creating minimal pairs of texts that differ only in the referenced group while maintaining comparable narrative conditions. To facilitate robust analysis, linguistic forms are generalized and associations between linguistic abstractions and groups are quantified using a variant of pointwise mutual information to detect expressions that appear disproportionately across groups. A fragment-ranking strategy then prioritizes text segments with a high concentration of biased linguistic signals, which allows for experts to assess the harmful potential of linguistic expressions in context, bridging quantitative analysis and qualitative interpretation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。