Purpose

To understand the meaning of a sentence, humans can focus on important words in the sentence, which reflects our eyes staying on each word in different gaze time or times. Thus, some studies utilize eye-tracking values to optimize the attention mechanism in deep learning models. But these studies lack to explain the rationality of this approach. Whether the attention mechanism possesses this feature of human reading needs to be explored.

Design/methodology/approach

The authors conducted experiments on a sentiment classification task. Firstly, they obtained eye-tracking values from two open-source eye-tracking corpora to describe the feature of human reading. Then, the machine attention values of each sentence were learned from a sentiment classification model. Finally, a comparison was conducted to analyze machine attention values and eye-tracking values.

Findings

Through experiments, the authors found the attention mechanism can focus on important words, such as adjectives, adverbs and sentiment words, which are valuable for judging the sentiment of sentences on the sentiment classification task. It possesses the feature of human reading, focusing on important words in sentences when reading. Due to the insufficient learning of the attention mechanism, some words are wrongly focused. The eye-tracking values can help the attention mechanism correct this error and improve the model performance.

Originality/value

Our research not only provides a reasonable explanation for the study of using eye-tracking values to optimize the attention mechanism but also provides new inspiration for the interpretability of attention mechanism.

中文翻译：

---

注意机制是否具有人类阅读的特征？情感分类任务的一个视角

目的

为了理解一个句子的意思，人类可以将注意力集中在句子中的重要词上，这反映了我们的眼睛在不同的凝视时间或时间上停留在每个词上。因此，一些研究利用眼动追踪值来优化深度学习模型中的注意力机制。但这些研究缺乏解释这种方法的合理性。注意力机制是否具有人类阅读的这种特征需要探索。

设计/方法/途径

作者对情感分类任务进行了实验。首先，他们从两个开源眼动追踪语料库中获取眼动值来描述人类阅读的特征。然后，从情感分类模型中学习每个句子的机器注意力值。最后，进行了比较以分析机器注意力值和眼动追踪值。

发现

通过实验，作者发现注意力机制可以关注重要的词，如形容词、副词和情感词，这些词对于情感分类任务中句子的情感判断具有重要价值。它具有人类阅读的特点，阅读时重点关注句子中的重要单词。由于对attention机制的学习不足，导致部分词被错误聚焦。眼动值可以帮助注意力机制纠正这个错误并提高模型性能。

原创性/价值

我们的研究不仅为利用眼动追踪值优化注意力机制的研究提供了合理的解释，也为注意力机制的可解释性提供了新的启发。