In this article, we present our methodologies for SemEval-2021 Task-4: Reading Comprehension of Abstract Meaning. Given a fill-in-the-blank-type question and a corresponding context, the task is to predict the most suitable word from a list of 5 options. There are three sub-tasks within this task: Imperceptibility (subtask-I), Non-Specificity (subtask-II), and Intersection (subtask-III). We use encoders of transformers-based models pre-trained on the masked language modelling (MLM) task to build our Fill-in-the-blank (FitB) models. Moreover, to model imperceptibility, we define certain linguistic features, and to model non-specificity, we leverage information from hypernyms and hyponyms provided by a lexical database. Specifically, for non-specificity, we try out augmentation techniques, and other statistical techniques. We also propose variants, namely Chunk Voting and Max Context, to take care of input length restrictions for BERT, etc. Additionally, we perform a thorough ablation study, and use Integrated Gradients to explain our predictions on a few samples. Our best submissions achieve accuracies of 75.31% and 77.84%, on the test sets for subtask-I and subtask-II, respectively. For subtask-III, we achieve accuracies of 65.64% and 62.27%.

中文翻译：

---

LRG在SemEval-2021上的任务4：使用增强，语言特征和投票来提高抽象单词的阅读理解能力

在本文中，我们介绍了SemEval-2021任务4：对抽象含义的阅读理解的方法。给定一个空白的问题和相应的上下文，任务是从5个选项的列表中预测最合适的单词。此任务中包含三个子任务：不可感知性（子任务I），非专一性（子任务II）和交集（子任务III）。我们使用经过屏蔽语言建模（MLM）任务预训练的基于变压器的模型的编码器来构建空白填充（FitB）模型。此外，为了对不可感知性进行建模，我们定义了某些语言特征，并且为了对非特定性进行建模，我们利用了词汇数据库提供的上位词和下位词的信息。具体来说，对于非特异性，我们尝试了增强技术和其他统计技术。我们还提出了变体，分别是Chunk Voting和Max Context，以照顾BERT的输入长度限制等。此外，我们进行了彻底的消融研究，并使用Integrated Gradients解释了对一些样本的预测。在子任务I和子任务II的测试集上，我们最好的提交分别达到了75.31％和77.84％的准确性。对于子任务III，我们实现了65.64％和62.27％的准确度。