An indexing system for the relevance of academic production and research from digital repositories and metadata

This paper aims to propose a set of quantitative statistical indicators for measuring the scientific relevance of research groups and researchers, based on high-impact open-access digital production repositories.

An action research (AR) methodology is proposed in which research is associated with the practice; research informs practice and practice is responsible for informing research in a cooperative way. AR is divided into five phases, beginning with the definition of the problematic scenario and an analysis of the state of the art and ending with conducting tests and publishing the results.

The proposed indicators were used to characterise group and individual output in a major public university in south-eastern Mexico. University campuses hosting a large number of high-impact research groups. These indicators were very useful in generating information that confirmed specific assumptions about the scientific production of the university.

The data used here were retrieved from Scopus and open access national repository of Mexico. It would be possible to use other data sources to calculate these indicators.

The system used to implement the proposed indicators is independent of any particular technological tool and is based on standards for metadata description and exchange, thus facilitating the easy integration of new elements for evaluation.

Many organisations evaluate researchers according to specific criteria, one of which is the prestige of journals. Although the guidelines differ between evaluation bodies, relevance is measured based on elements that can be adapted and where some have greater weight than others, including the prestige of the journal, the degree of collaboration with other researchers and individual production, etc. The proposed indicators can be used by various entities to evaluate researchers and research groups. Each country has its own organisations that are responsible for evaluation, using various criteria based on the impact of the publications.

The proposed indicators assess based on the importance of the types of publications and the degree of collaborations. However, they can be adapted to other similar scenarios.