用核方法评估生物序列模型，发现主流蛋白设计模型存在拟合偏差。
Kernel-Based Evaluation of Conditional Biological Sequence Models
- 提出ACMMD核测度，无偏估计模型与真实分布的差异
- 发现ProteinMPNN在多个蛋白家族中无法拟合数据
- 可调温度参数提升模型可靠性，适合生物序列建模研究者
我们提出一套基于核的方法，用于评估条件序列模型的设计并优化超参数，聚焦计算生物学问题。核心是新提出的增强条件最大均值差异（ACMMD），衡量真实条件分布与模型估计之间的差距。只要模型可采样，就能从数据中无偏估计ACMMD，量化模型绝对拟合度，集成于假设检验，并评估模型可靠性。通过分析流行的蛋白设计模型ProteinMPNN，我们发现其在多个蛋白家族中无法拟合数据，且可通过调节温度超参数改善拟合效果。
原文摘要 · Abstract (English)
We propose a set of kernel-based tools to evaluate the designs and tune the hyperparameters of conditional sequence models, with a focus on problems in computational biology. The backbone of our tools is a new measure of discrepancy between the true conditional distribution and the model's estimate, called the Augmented Conditional Maximum Mean Discrepancy (ACMMD). Provided that the model can be sampled from, the ACMMD can be estimated unbiasedly from data to quantify absolute model fit, integrated within hypothesis tests, and used to evaluate model reliability. We demonstrate the utility of our approach by analyzing a popular protein design model, ProteinMPNN. We are able to reject the hypothesis that ProteinMPNN fits its data for various protein families, and tune the model's temperature hyperparameter to achieve a better fit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。