Ensuring accurate resource identification

Nature Protocols is pleased to be a part of the Resource Identification Initiative, a project aimed at improving the reproducibility of research by clearly identifying key biological resources. Stable unique digital identifiers, called Research Resource Identifiers (RRIDs), are assigned to individual resources, allowing users to accurately identify and source them, track their history, identify known problems (such as cell line contamination and misidentification) and find relevant research papers. Following a successful 6-month trial, we will now require authors to provide RRIDs for all antibodies and cell lines used in their protocols. We will also be encouraging them to add RRIDs for other tools (such as plasmids and organisms) where they think this is helpful.

RRIDs were launched in 2014 by a FORCE 11 group, which set up a preliminary trial with 25 journals and reported their findings in a F1000R paper¹. Approximately 670 journals now encourage the use of RRIDs (including the Springer Nature, Cell Press, PLoS and AACR journals), but uptake by authors has been inconsistent. Six months ago, we set up a trial in which we required authors to add RRIDs for all the cell lines and antibodies used in their protocols unless they had an exceptional reason for not doing so. This trial has now finished, and the results are very encouraging: a 97.3% compliance rate (compared with 5% before the trial) and overwhelmingly positive feedback.

This is unsurprising, as the advantages for authors and readers are clear. RRIDs make it easier to identify and source the correct resources so that less time is spent hunting them down (particularly for cell lines that have been gifted and do not have catalog numbers) and, more importantly, less time is wasted in the lab using incorrect resources. RRIDs also make it easier to track the history of resources (e.g., the transfer of antibodies between vendors) and to identify known problems, such as cell lines that have been misidentified or contaminated. Other useful information, such as cell doubling time and HLA typing info, is also provided. Moreover, each RRID has a ‘usage and citation metrics’ link that allows authors and readers to find other groups using the same resource and set up collaborations, request advice on handling the resource, compare experimental data, etc. For example, our recently published protocol on engineering artificial microRNA clusters for gene therapy uses HEK293 cells², which have been assigned the RRID CVCL_0045. This RRID entry includes links to 316 other PubMed-listed articles that have also used this cell line and cited the RRID.

The system is also easy to use. All RRIDS can be found at a single website (http://scicrunch.org/resources) managed by SciCrunch, an open-access data sharing and display platform. Each RRID on this website links directly to the authoritative databases for each resource type (e.g., Cellosaurus or NCBI BioSample for cell lines, the Antibody Registry for antibodies, and the Rat Genome Database (RGD), Mouse Genome Informatics (MGI), the Zebrafish Information Network (ZFIN), Wormbase or Flybase for model organisms). When you cannot find an RRID for your resource, you can help this initiative by creating a new one, simply by clicking on the ‘Add a resource’ button at the bottom of the RRID homepage and following the instructions, which vary depending on the resource. For example, RRIDs for cell lines are added by emailing Cellosaurus with the relevant details, whereas antibodies are added by completing a form on the Antibody Registry site. All requests for new entries are checked by a curator, and duplicated entries are resolved, but the process moves quickly (within 1–2 working days) and so should not cause delays with publication.

After completion of the trial, we used a tool called SciScore (developed by SciCrunch) to check the integrity of the author-assigned RRIDs. SciScore uses natural-language processing and machine learning to evaluate compliance with well-established reproducibility guidelines (such as the NIH guidelines and Nature Reporting Summary) and checks for key resource identifiability for various resource types³. The results were very promising, with 100% accuracy for both antibodies and cell lines. We also asked authors about their experiences of using the system through surveys sent to all participants asking for their feedback. From the responses we received, the average overall rating for using the system was 4.75 out of 5. Most authors (83%) had no problems using it, and those who did need help were impressed by the service they received from the helpdesk (average rating of 4.75 out of 5). Authors told us that the helpdesk was “very quick to respond” and “the registration process was very smooth.” All authors who added RRIDs agreed that they would increase the reproducibility of their protocol, and one author told us, “We believe that RRIDs might be a big help for the community.”

For the trial, we asked authors to provide RRIDs only for antibodies and cell lines, as the benefits are clearer (these resources are notoriously difficult to identify) and the challenges are likely to be fewer than for other resources, particularly organisms. The F1000 study showed lower error rates when RRIDs were assigned to antibodies (4%) compared with organisms (13%)¹. However, our feedback survey showed strong support for expanding our policy to include other resources (such as plasmids, organisms and computational tools), so we will now be encouraging authors to add RRIDs for these too. It is important to note that RRIDs will not act as a replacement for other resource details—all other details (such as the name, supplier, catalog number and reagent composition) must still be provided—and that our new policy will enhance our existing policies relating to cell line authentication and deposition, not replace them. We are constantly striving to make our protocols as reliable and reproducible as possible, and we hope this new initiative will make life easier for you, our readers.