仅用访谈文本识别视频中的犹豫与矛盾，性能远超视觉模型。
TellTale: Blending Multi-Instance LoRA Text Encoders and a Zero-Shot LLM Judge for Ambivalence/Hesitancy Recognition in Videos
- 用多实例学习融合两个微调的文本编码器和一个零样本大模型判断犹豫。
- 在152个未见参与者视频上达0.7364宏平均F1，远超视觉基线的0.2827。
- 无需视频或音频，适合隐私敏感场景下的情绪分析应用。
我们提出TellTale，一种仅依赖访谈转录文本的犹豫/矛盾（A/H）识别方法，基于第三届A/H视频识别挑战赛（第11届ABAW研讨会，ECCV 2026）的BAH数据集进行评估。尽管数据集包含视频、音频、人脸片段和转录文本，但TellTale仅使用转录文本，融合三种概率流。两个文本编码器multilingual-e5-large和mDeBERTa-v3-base通过参数高效的LoRA适配器，在多实例学习（MIL）目标下微调，其中转录段落独立打分，并以平滑最大值聚合，仅需视频级标签监督。第三种流无需训练：一个量化后的140亿参数指令型大模型被零样本提示，对每段转录文本评分A/H程度。三种概率通过加权平均和单一决策阈值合并，阈值由按参与者分组的交叉验证预测选择。在主办方评分的私有测试集（152个未见参与者视频）上，TellTale取得0.7364的宏平均F1和0.7940的平均精度，而官方视觉基线仅为0.2827。
原文摘要 · Abstract (English)
We present TellTale, a text-only approach to ambivalence/hesitancy (A/H) recognition in interview videos, evaluated on the BAH dataset as part of the 3rd A/H Video Recognition Challenge (11th ABAW Workshop, ECCV 2026). Although the dataset provides video, audio, facial crops, and transcripts, TellTale relies on the transcript alone and combines three probability streams. Two text encoders, multilingual-e5-large and mDeBERTa-v3-base, are fine-tuned with parameter-efficient LoRA adapters under a multiple-instance learning (MIL) objective, in which transcript chunks are scored individually and pooled with a smooth maximum so that only the video-level label is needed for supervision. The third stream requires no training: a quantized 14B instruction LLM is prompted, zero-shot, to rate each transcript for A/H. The three probabilities are combined by a weighted average and a single decision threshold, both selected on participant-grouped cross-validated predictions. On the organizer-scored private test set of 152 videos from unseen participants, TellTale achieves a Macro-F1 of 0.7364 and an average precision of 0.7940, compared with 0.2827 Macro-F1 for the official vision-based baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。