Scientific papers are complex and understanding the usefulness of these papers requires prior knowledge. Peer reviews are comments on a paper provided by designated experts on that field and hold a substantial amount of information, not only for the editors and chairs to make the final decision, but also to judge the potential impact of the paper. In this paper, we propose to use aspect-based sentiment analysis of scientific reviews to be able to extract useful information, which correlates well with the accept/reject decision. While working on a dataset of close to 8k reviews from ICLR, one of the top conferences in the field of machine learning, we use an active learning framework to build a training dataset for aspect prediction, which is further used to obtain the aspects and sentiments for the entire dataset. We show that the distribution of aspect-based sentiments obtained from a review is significantly different for accepted and rejected papers. We use the aspect sentiments from these reviews to make an intriguing observation, certain aspects present in a paper and discussed in the review strongly determine the final recommendation. As a second objective, we quantify the extent of disagreement among the reviewers refereeing a paper. We also investigate the extent of disagreement between the reviewers and the chair and find that the inter-reviewer disagreement may have a link to the disagreement with the chair. One of the most interesting observations from this study is that reviews, where the reviewer score and the aspect sentiments extracted from the review text written by the reviewer are consistent, are also more likely to be concurrent with the chair's decision.

中文翻译：

---

科学评论的基于方面的情绪分析

科学论文很复杂，理解这些论文的用处需要先验知识。同行评审是对该领域指定专家提供的论文的评论，包含大量信息，不仅可供编辑和主席做出最终决定，还可以判断论文的潜在影响。在本文中，我们建议使用科学评论的基于方面的情感分析来提取有用的信息，这些信息与接受/拒绝决策密切相关。在处理来自机器学习领域顶级会议之一 ICLR 的接近 8k 条评论的数据集时，我们使用主动学习框架构建了一个用于方面预测的训练数据集，该数据集进一步用于获取方面和情绪对于整个数据集。我们表明，对于接受和拒绝的论文，从评论中获得的基于方面的情绪分布有显着差异。我们使用这些评论中的方面情绪来进行有趣的观察，论文中出现的某些方面以及评论中讨论的某些方面强烈地决定了最终的建议。作为第二个目标，我们量化了审稿人之间的分歧程度。我们还调查了审稿人和主席之间的分歧程度，发现审稿人之间的分歧可能与与主席的分歧有关。这项研究中最有趣的观察结果之一是，评论者的评分和评论者撰写的评论文本中提取的方面情绪是一致的，