为眼动数据设计新分词方法，让大模型能更好理解和生成眼动轨迹。
Tokenization of Gaze Data
- 提出五种眼动数据分词策略，评估其重建与压缩能力。
- 分位数分词在预测眼动位置上最优，聚类分词在速度预测上表现最佳。
- 首次实现用预训练多模态模型处理眼动数据，适合人机交互研究者。
当前大型语言模型（LLM）和多模态大语言模型（MLLM）的性能很大程度上依赖于分词策略。尽管文本和视觉输入的分词已得到广泛研究，但眼动数据因自身特性尚未有相关研究。然而，建立合适的分词策略可使预训练的多模态大模型（如通过微调）利用其视觉理解能力处理眼动数据。本文针对这一研究空白，分析了五种不同的眼动数据分词方法，在三个不同数据集上评估其在眼动轨迹预测与生成任务中的表现。我们从重建精度和压缩效率两个维度评估分词器性能，并为每种分词策略训练一个LLM，比较其生成与预测能力。结果表明：分位数分词在预测眼动位置方面表现最佳，而k-means分词在预测眼动速度时最优。
原文摘要 · Abstract (English)
A considerable part of the performance of today's large language models (LLM's) and multimodal large language models (MLLM's) depends on their tokenization strategies. While tokenizers are extensively researched for textual and visual input, there is no research on tokenization strategies for gaze data due to its nature. However, a corresponding tokenization strategy would allow using the vision capabilities of pre-trained MLLM's for gaze data, for example, through fine-tuning. In this paper, we aim to close this research gap by analyzing five different tokenizers for gaze data on three different datasets for the forecasting and generation of gaze data through LLMs (cf.~\cref{fig:teaser}). We evaluate the tokenizers regarding their reconstruction and compression abilities. Further, we train an LLM for each tokenization strategy, measuring its generative and predictive performance. Overall, we found that a quantile tokenizer outperforms all others in predicting the gaze positions and k-means is best when predicting gaze velocities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。