Over the past decade, challenges regarding reproducibility in science have come to the forefront and tools to improve it are being developed in several areas. The discussion focuses on many aspects, such as the need to address the crisis from the perspective of antibody validation1,2, cell line authentication3,4 and, more recently, artificial intelligence5,6. This has led to challenges in science as the problem is immense7,8 and solutions to improve the situation are complex9,10. Community-driven initiatives to improve reproducibility by developing standards for antibody validation11 and cell line authentication12,13,14 have created more awareness around the problem and have given researchers the tools they need to start to solve it. This has increased the dialog among scientists and, ultimately, will lead to more solutions for reproducibility.

The hardware and software components of light microscope systems are complex and vary widely from one to another. Many light microscopy–based imaging modalities generate complex and multidimensional sets of images (for example, in three dimensions, multiple colors, millions of pixels). This complexity makes the development of generalized and widely accepted protocols and guidelines for quality control (QC) in light microscopy experiments challenging. Many individual efforts have aimed to improve the situation in light microscopy, but up until now there has been no concerted effort by the global light microscopy community.

However, the time is right to take action. Researchers are increasingly using light microscopy to publish quantitative data rather than for qualitative observations. Therefore, it is vital that the performance and limitations of the microscope systems used are routinely measured and fully understood. Only with this knowledge can the data captured be reliable, robust, accurate and reproducible. Management of QC is essential and should be mandatory for both spatial measurements (for example, morphology, size, distance) and quantitative fluorescence intensity measurements (for example, expression level, protein activity, local concentration) to ensure microscopes are calibrated and stable over time so as to safeguard repeatability and accuracy of measurements. Moreover, imaging data are becoming more broadly shared for community analysis and made available in public image data repositories15,16. If image data are broadly shared, it is essential that information about how the data were generated is well documented and that images are of high quality. Therefore, the equipment upon which the images are captured should be subjected to robust QC standards.

Many meetings and discussions around the topic of microscopy QC and standards have taken place over the last decade at various venues; for example, the Focus on Microscopy, German BioImaging, Global BioImaging Exchange of Experience, BioImaging UK Light Microscopy Facility, Association of BioMolecular Resource Facilities (ABRF) and European Light Microscopy Initiative (ELMI) meetings. However, these community efforts have not been coordinated at the global level, have not focused on accepted guidelines from the International Organization for Standardization (ISO), and have not been through rigorous community-driven peer review.

In 2019, as part of the Core Facility Day at the ELMI meeting, a survey was launched and completed by 225 microscopists from around the world about common practices for the management of microscope QC. The comprehensive survey highlighted that microscope QC procedures across the community are inconsistent in their nature and frequency. Facilities perform different quality tests, use dozens of different standard samples and software tools, metrics and protocols, and perform checks on different timelines (for example, weekly, monthly, annually) or not at all. The main barrier to frequent in-depth quality checks and standardization was the lack of human resources to perform them. They are typically done manually, take a considerable amount of time and tie up both imaging facility staff time and instruments. In addition, other barriers were identified as a lack of widely adopted and agreed-upon guidelines, access to and consensus for standard samples and protocols, and robust training for how to perform QC tests.

In 2018, Global BioImaging published “Common international recommendation for quality assurance and management in open access imaging infrastructures” (https://globalbioimaging.org/documents). However, these are general guidelines that do not include in-depth protocols. More recently, ISO published the first international standard for confocal microscopy: “ISO 21073:2019 – Confocal microscopes — optical data of fluorescence confocal microscopes for biological imaging.” On the basis of that standard, BioImaging UK has recently published a companion paper providing protocols and recommended tools to implement ISO 21073:2019, entitled “Interpretation of Confocal ISO 21073: 2019 confocal microscopes: optical data of fluorescence confocal microscopes for biological imaging – recommended methodology for quality control”17.

QUAREP-LiMi formation

The ELMI survey and the challenges it highlighted demonstrated a clear need for a global joint initiative involving microscopists from different fields and other key stakeholders including imaging scientists, image analysts, standards organizations, microscope manufacturers, funding organizations and publishers. In response, a community-driven initiative entitled Quality Assessment and Reproducibility for Instruments & Images in Light Microscopy (QUAREP-LiMi) was formed in April 2020. It aims to improve reproducibility for light microscopy image data through QC management of instruments and images. Its ultimate goal is to agree on a common set of QC guidelines for calibrating hardware and acquiring and managing microscope images and related software. The tangible outcomes of QUAREP-LiMi include protocols, metadata models and tools (automated if possible) to ensure reproducible, reliable, sharable and easily searchable images and scientific research results. As of 6 May 2021, QUAREP-LiMi is made up of 269 individuals from 25 countries, with members from academia, industry, standards organizations and scientific publishers. Many members are drawn from well-established national and international networks, including but not limited to German BioImaging (GerBI), Microscopie de Fluorescence Multidimensionnelle (RT-MFM), BioImaging UK, the Royal Microscopical Society (RMS), Euro-BioImaging ERIC, Global BioImaging, and BioImaging North America (BINA).

Working groups

The QUAREP-LiMi initiative is organized into a number of focused working groups (WGs) that address areas or topics important for microscope hardware and image data QC management, including data analysis and presentation. Figure 1 presents the major topics and aims covered by the QUAREP-LiMi initiative in their WGs. Their aims are to drive consensus on the use of standard samples and software tools and agree on the metrics to be measured and reported. They will develop and publish training and standard operating procedures to ensure wide adoption by microscope custodians and users. The overarching aim of the WGs is to develop and recommend straightforward yet comprehensive protocols and robust samples that can be used to streamline and eventually automate the QC process.

Fig. 1: Overview of topics and aims covered by QUAREP-LiMi initiative.
figure 1

QUAREP-LiMi aims to define standard operating procedures and tools for device monitoring, benchmarking and quality management of light microscopy instruments and images that are aligned with existing ISO standards, as well as to develop in liaison with ISO or other stakeholders (corporate partners, funders and publishers) a community-based consensus around microscope and image data QC.

Full size image

QUAREP-LiMi currently comprises 11 WGs, each led by a chair and co-chair. Each WG is formed from members of the global microscopy community. WGs meet virtually, typically once a month, and their agendas, minutes and current working protocols can be found on the QUAREP-LiMi webpage (https://quarep.org/). Final protocols and guidelines will be published in journals on an open-access basis. Activities now focus on widefield and confocal laser scanning microscopy platforms but will later be expanded to other imaging modalities. WGs 1 to 6 will acquire image data at multiple laboratories to test the reproducibility of samples and data acquisition and analysis tools to develop robust protocols.

WG1: Illumination Power

Focus. Metrics and tools to measure microscope light source power and stability on different time scales.

Current activities. Establish a protocol for measuring illumination power and stability during both short- and long-term image acquisition sessions, using calibrated external power sensors.

WG2 Detection System Performance

Focus. Metrics and tools to measure and report detection system (for example, camera, photomultiplier tube, avalanche photodiode) performance.

Current activities. Standardize the characterization of detection system performance (including the emission light path) and create accepted procedures and protocols for monitoring it over time. Define universal, externally measurable parameters applicable to any type of detection system.

WG3 Uniformity of Illumination Field – Flatness

Focus. Define a set of protocols, tools and guidelines to assess the uniformity of illumination across the microscope field of view and allow correction.

Current activities. Develop protocols and tools based on a consensus, to measure and correct field non-uniformity in single images or tiles of images of a large sample that have been stitched together.

WG4 System Chromatic Aberration and Co-Registration

Focus. Metrics and tools to measure chromatic shifts in x, y and z and protocols to allow co-registration correction.

Current activities. Use multicolored beads or similar preparations to measure co-registration accuracy and develop protocols to correct for chromatic aberrations and align images in x, y and z.

WG5 Lateral and Axial Resolution

Focus. Metrics and tools to measure and report lateral and axial microscopy resolution limits.

Current activities. Develop sample preparation, image acquisition and data analysis protocols for samples of sub-resolution fluorescent beads or similar preparations for monitoring resolution over time.

WG6 Stage and Focus – Precision and Other

Focus. Metrics and tools to measure and report stability and precision of motorized stage platforms, including sample holders and microscope focus drives.

Current activities. Define key terms and develop measurement standards, testing protocols and performance benchmarks to evaluate xyz movement in terms of stability, reproducibility and repeatability.

WG7 Microscopy Data Provenance and QC Metadata

Focus. Develop guidelines defining what ‘data provenance’ and QC metadata should be reported for distinct types of imaging data.

Current activities. The 4D Nucleome (4DN) Imaging WG and the BINA Quality Control and Data Management WG have developed a tiered set of microscopy metadata guidelines and a suite of extensions of the Open Microscopy Environment (OME) data model that scale with experimental complexity and requirements. These are tailored to enhance comparability and reproducibility in light microscopy. WG7 aims to establish a coordinated outreach strategy to achieve a wide community consensus around the proposed metadata specifications.

WG8 White Papers

Focus. Publish white papers to communicate and seek cooperation from the microscopy community to raise awareness and promote QUAREP-LiMi’s short- and long-term goals.

Current activities. The first white paper is published on arXiv18. The WG is now focused on raising awareness about the white paper and QUAREP-LiMi with various stakeholders to support the work of this initiative, including (1) prospective new members, (2) imaging scientists and bioimage analysts, (3) group leaders and principal investigators, (4) research scientists, (5) scientific publishers, (6) leads (CEOs and directors) of companies and commercial application specialists, and (7) prospective funders.

WG9 Overall Planning and Funding

Focus. Coordinate and promote QUAREP-LiMi, seek funding opportunities and engage and liaise with stakeholders.

Current activities. Formalize publication and authorship guidelines; engage with corporate partners, standardization organizations, scientific publishers and funding bodies. Develop and update webpage, tools database and tools to keep WGs organized and running efficiently.

Long-term goals. (1) Ensure that the output of QUAREP-LiMi achieves maximum impact; (2) seek buy-in from microscope manufacturers; (3) obtain funding from national bodies, scientific publishers and learned societies; (4) keep stakeholders informed and share information; and (5) coordinate all WGs and future meetings.

WG10 Image Quality

Focus. Define image quality parameters and their weighted impact based on experiment types and microscope modalities to create tools to evaluate the quality of individual images.

Current activities. Define weighted image quality parameters and assign experiment- and microscope-specific QC rating for individual images.

Long-term goals. Integration of community agreed-upon image quality metrics in image metadata.

WG11 Microscopy Publication Standards

Focus. Develop guidelines and best practices to ensure quality microscope metadata and microscopy methods reporting.

Current activities. (1) Inform scientific publishers of methods reporting standards and align them with the recommendations of QUAREP-LiMi; (2) facilitate the involvement of technical reviewers for microscopy-based data; (3) promote and increase the appropriate acknowledgment and co-authorship of imaging scientists and imaging facilities in publications; (4) encourage publishers to compel authors to make raw imaging data available; and (5) propose minimum standards for microscope-based figure quality.

Stakeholders and beneficiaries

QUAREP-LiMi comprises many stakeholders that are beneficiaries of more rigorous and reproducible microscopy image data and also need to be part of the solution. Stakeholders include (1) microscope users and custodians (facility and non-facility), who can be assured of high-quality image data and access to community-developed and agreed-upon guidelines, recommendations, tools and protocols; (2) researchers, who will benefit from well-maintained microscopes and the ability to reproduce data from the literature and collaborators; (3) scientific publishers, who will see improved quality of microscopy data upon which scientific conclusions are based and publications that are more reliable for other researchers, leading to more significant scientific discovery and greater trust in scientific output; (4) funding bodies, who will be rewarded with a better return on their investments due to higher data quality and will also benefit from data sharing and consequent new discoveries without the need to repeat costly experiments due to low quality; (5) microscope manufacturers, who will benefit from a better knowledge of the instruments’ performance in the field, allowing predictive instrument service and technical improvements for future instrument development; and (6) standards organizations, who can work efficiently and effectively with the global community to gain consensus, develop quality standards and be part of the solution through promotion and implementation.

Invitation to join QUAREP-LiMi

The QUAREP-LiMi initiative depends on the input of the international microscopy community. This includes academics, industry, funders, standards agencies and scientific publishers. Joint development of community-driven recommendations and guidelines is essential for them to be accepted and adopted by the majority of microscopists. For more in-depth information about the QUAREP-LiMi initiative, please see the white paper18. If you are interested in actively working in the QUAREP-LiMi initiative to improve the quality of light microscopy imaging, please sign up at https://quarep.org/contact/.