Due to its quantitative nature, bibliometrics is becoming increasingly popular among policy makers for academic hiring and career promotions. In this article, we quantitatively assess the impact that the granularity level in the classification of scientific areas would entail on research evaluation based on bibliometric indicators. We use as a case study the Italian national habilitation system (ASN), which classifies faculty members according to their academic discipline and relies on journal counts, citations, and h-indices as a basis for promoting tenure track researchers to associate professors and associate to full professors. The assessment checks whether the individual indicators of a researcher are above a certain threshold, e.g., the median over the population of researchers working in the same discipline. Our investigation focuses on two related, rather broad disciplines: computer science and computer engineering. We show that the ASN practice of using the same thresholds for all members of a scientific discipline can favor certain sub-communities that are characterized by higher bibliometric indicators, and disfavor others. We report evidence that up to 30% of Italian faculty members of certain sub-communities would see their indicators drop below the threshold, thus becoming not eligible for promotion, if the ASN were conducted on a more accurate, fine-grained classification. Conversely, in the same scenario, up to 11% of faculty members, in different sub-communities, would see their indicators rise above the threshold, granting them eligibility. Our data set includes 1685 authors, 89,185 distinct publications, and 262,286 author-publication pairs.

中文翻译：

---

学术推广中的文献计量学：意大利计算机科学与工程的案例研究

由于其定量性质，文献计量学在学术招聘和职业晋升的决策者中越来越受欢迎。在本文中，我们定量评估了科学领域分类的粒度级别对基于文献计量指标的研究评估的影响。我们使用意大利国家职业资格系统 (ASN) 作为案例研究，该系统根据教职人员的学科进行分类，并依靠期刊数量、引文和 h 指数作为将终身教职研究人员晋升为副教授和副教授的基础全教授。评估检查研究人员的个人指标是否高于某个阈值，例如，在同一学科工作的研究人员总体的中位数。我们的调查集中在两个相关且相当广泛的学科上：计算机科学和计算机工程。我们表明，对科学学科的所有成员使用相同阈值的 ASN 实践可能有利于某些具有较高文献计量指标的子社区，而不利于其他子社区。我们报告的证据表明，如果 ASN 以更准确、更细粒度的分类进行，则某些子社区中多达 30% 的意大利教职员工会看到他们的指标低于阈值，从而失去晋升资格。相反，在相同的情况下，不同子社区中多达 11% 的教职员工会看到他们的指标高于阈值，从而授予他们资格。我们的数据集包括 1685 位作者、89,185 篇不同的出版物和 262、