裸分类器反演在概念瓶颈文本生成中会失效，因代码脱离训练分布。
The Illusion of Control: Why Bare Classifier Inversion Silently Fails in Concept-Bottleneck Text Generation
- 用分类器反演生成概念码，但缺乏语言流畅性约束
- 所有反演方法均不如事后拟合的先验模型，最差时仅随机猜测
- 问题根源是代码偏离训练数据流形，适合评估可控生成方法的研究者
概念瓶颈可控生成通过低维概念码实现多属性控制，部署时需从目标属性配置合成概念码。本文在多轴组合泛化场景下，对比三种推理时生成码的方法：针对编码器头部的分类器反演、参考文本编码和事后标签条件先验。由于概念码无直接语言模型流畅性项，反演必须通过正则化约束其接近编码器训练分布。我们测试了裸反演及三种正则化变体：标签无关与标签相关的马氏距离惩罚，以及条件归一化流密度基线。结果表明，所有反演方法在三个骨干模型（参数量124M至8B）上均劣于对相同检查点拟合的事后先验模型。裸反演甚至悄然退化为随机猜测，可被直接测量的离流形码所解释。该诊断在真实世界基准和外部评估中得到验证，支持与已有基线的公平比较。
原文摘要 · Abstract (English)
Concept-bottleneck controllable generation routes multi-attribute control through a low-dimensional concept code that, at deployment, must be synthesised from a target attribute configuration. We study this problem in concept-bottleneck text generation under multi-axis compositional generalisation, comparing three ways to obtain the inference-time code: classifier inversion against the encoder heads, reference-text encoding, and a post-hoc label-conditioned prior. Since a concept code admits no direct LM-fluency term, regularising inversion must instead constrain the code toward the encoder's training distribution. We therefore test bare inversion and three regularised variants: label-agnostic and label-conditioned Mahalanobis penalties, and a conditional normalising-flow density baseline. Every inversion variant we test underperforms a simple post-hoc prior fitted to per-combination encoder means on the same checkpoints, across three backbone families spanning $124$M to $8$B parameters. The bare form of classifier inversion also silently collapses to chance, traceable to a directly measured off-manifold code. We validate this diagnosis on real-world benchmarks and under external evaluators, enabling fair comparison with published baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。