Purpose

Social comments are rich in information and useful in evaluating, ranking or retrieving different kinds of materials. However, their merits in representing or providing added values to scientific articles have not yet been studied. Therefore, the present study investigates the informativeness of open review reports as a kind of social comments in a scholarly setting.

Design/methodology/approach

A test collection was built consisting of 100 randomly selected queries, 1,962 reviewed documents and their reviewers' open reports from F1000Research. They were analyzed using natural language techniques. The comments' salient words were compared to the documents' and also to the Medical Subject Headings (MeSH) salient words. The receiver operating characteristic (ROC) curve was used to test the accuracy of the comments in representing their related articles.

Findings

The papers' contents and comments have a considerable number of salient words in common. The comments' salient words are also largely found in the MeSH, signifying their consistency with the knowledge tree and their potential to add some complementary features to their related items. The ROC curves confirm the accuracy of the comments in retrieving their related papers.

Originality/value

This research is the first to reveal the merits of open review reports on scientific papers, in terms of their relatedness to their mother articles, in specific, and to the knowledge tree, in general. They are found informative in not only representing the reviewed papers but also in adding values to the contents of the papers.

中文翻译：

---

确定评论的信息性：F1000Research的自然语言研究开放式同行评审报告

目的

社会评论内容丰富，可用于评估，排名或检索各种材料。但是，尚未研究它们在代表科学文章或为科学文章提供附加值方面的优点。因此，本研究调查公开审查报告作为一种学术环境中的社会评论的信息性。

设计/方法/方法

建立了一个测试集合，其中包含100个随机选择的查询，1,962个已审阅文档以及F1000Research的审阅者公开报告。他们使用自然语言技术进行了分析。将注释的显着词与文档以及医学主题词（MeSH）显着词进行比较。接收者操作特征（ROC）曲线用于测试代表其相关文章的评论的准确性。

发现

论文的内容和评论有很多共同点。注释的重要词还可以在MeSH中找到，这表明它们与知识树保持一致，并且有可能在相关项目中添加一些补充功能。ROC曲线确认了评论在检索其相关论文时的准确性。

创意/价值

这项研究是第一个揭示科学论文公开审阅报告的优点的方面，特别是它们与他们的母亲文章之间的关联性，以及与知识树之间的关联性。发现它们不仅可以提供经评审的论文，还可以为论文的内容增加价值。