Innovations in acquisition and analysis are transforming microscopy.

Despite substantial advances in microscope optics and computational resources, many microscopy experiments are still carried out much as they were decades ago, with samples being prepared and imaged one by one, zeroed in on by a trained user, and recorded in just enough numbers to be considered scientifically rigorous. Although this approach is undoubtedly powerful, it requires abundant hands-on time and expertise, can be limited in terms of statistics, and can be biased by the imaging of structures that match an expectation.

The ‘smart’ microscope will improve imaging. Credit: Marina Corral Spence/Springer Nature

There has been a shift away from fully manual imaging that is poised to eventually take humans out of the loop in imaging experiments. For example, technological developments in robotics for handling biological samples have made many types of experiments high throughput. Combined with software and hardware tools developed to automate high-throughput imaging, it is easy to imagine a world where human hands aren’t required for sample preparation, loading and image acquisition. Super-resolution single-molecule localization microscopy is one area where such automated strategies are beginning to bear fruit (Opt. Express 26, 30882–30900, 2018; Nat. Commun. 10, 1223, 2019; Nat. Methods 14, 1184–1190, 2017).

Automated acquisition has benefits beyond ease and throughout, and can lead to higher quality images. A core principle of ‘smart microscopy’ is that the microscope and acquisition controls interact with each other to create positive feedback. One notable example is the Autopilot framework for adaptive imaging, which has enabled time-lapse imaging of the development of zebrafish, fruit fly and mouse embryos (Nat. Biotechnol. 34, 1267–1278, 2016; Cell 175, 859–876.e33, 2018). In this case, the microscope control software actively manages aspects such as the position and 3D orientation of the sample and acquisition parameters in real time to optimize speed, quality and consistency of imaging over time.

Beyond acquiring images, several user-friendly tools are established for semi- and fully automated analysis for most standard tasks, including segmentation and phenotyping, and even more detailed quantitative analyses. These methods are under constant development and enable very sophisticated analyses of complex samples.

We think smart microscopy will be an important trend in years to come; innovation will be spurred by improvements to and seamless integration of all the aspects listed above. Deep machine learning is likely to be pivotal in such improvements, as well as in strategies for handling the data deluge associated with such imaging. We also hope that at the heart of such advances will be benefits to the health of samples, as rightly stressed in a Commentary on this topic (Nat. Biotechnol. 33, 815–818, 2015).