用自述内容检验算法对人的描述是否准确可靠。
Representation Fidelity:Auditing Algorithmic Decisions About Humans Using Self-Descriptions
- 通过对比外部输入与个人自述，评估算法决策依据的合理性。
- 发现3万条贷款申请中存在显著描述偏差，部分匹配度低于0.4。
- 提出首个可量化的评估框架，适合关注算法公平性的研究者使用。
本文提出以‘表示保真度’（Representation Fidelity）作为验证算法对人决策合理性的新维度。该方法通过测量同一人两种表征间的距离来评估：其一是决策所依赖的外部输入表征，其二是由被决策人提供的自述内容，仅用于验证输入表征的准确性。我们分析了两类表征间的差异性质，提出量化这些差异的方法，并构建了一个通用的表示错位类型学，以判断表示保真度水平。此外，本文基于德国信用数据集构建了首个评估基准——2025年贷款审批自述语料库（Loan-Granting Self-Representations Corpus 2025），包含30,000条合成的自然语言自述文本及其对应申请人表征，并由专家标注每对表征间的表示错位情况。
原文摘要 · Abstract (English)
This paper introduces a new dimension for validating algorithmic decisions about humans by measuring the fidelity of their representations. Representation Fidelity measures if decisions about a person rest on reasonable grounds. We propose to operationalize this notion by measuring the distance between two representations of the same person: (1) an externally prescribed input representation on which the decision is based, and (2) a self-description provided by the human subject of the decision, used solely to validate the input representation. We examine the nature of discrepancies between these representations, how such discrepancies can be quantified, and derive a generic typology of representation mismatches that determine the degree of representation fidelity. We further present the first benchmark for evaluating representation fidelity based on a dataset of loan-granting decisions. Our Loan-Granting Self-Representations Corpus 2025 consists of a large corpus of 30 000 synthetic natural language self-descriptions derived from corresponding representations of applicants in the German Credit Dataset, along with expert annotations of representation mismatches between each pair of representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。