Volumetric imaging of samples using fluorescence microscopy plays an important role in various fields including physical, medical and life sciences. Here we report a deep learning-based volumetric image inference framework that uses 2D images that are sparsely captured by a standard wide-field fluorescence microscope at arbitrary axial positions within the sample volume. Through a recurrent convolutional neural network, which we term as Recurrent-MZ, 2D fluorescence information from a few axial planes within the sample is explicitly incorporated to digitally reconstruct the sample volume over an extended depth-of-field. Using experiments on C. elegans and nanobead samples, Recurrent-MZ is demonstrated to significantly increase the depth-of-field of a 63×/1.4NA objective lens, also providing a 30-fold reduction in the number of axial scans required to image the same sample volume. We further illustrated the generalization of this recurrent network for 3D imaging by showing its resilience to varying imaging conditions, including e.g., different sequences of input images, covering various axial permutations and unknown axial positioning errors. We also demonstrated wide-field to confocal cross-modality image transformations using Recurrent-MZ framework and performed 3D image reconstruction of a sample using a few wide-field 2D fluorescence images as input, matching confocal microscopy images of the same sample volume. Recurrent-MZ demonstrates the first application of recurrent neural networks in microscopic image reconstruction and provides a flexible and rapid volumetric imaging framework, overcoming the limitations of current 3D scanning microscopy tools.

使用荧光显微镜对样品进行体积成像在物理、医学和生命科学等各个领域发挥着重要作用。在这里，我们报告了一种基于深度学习的体积图像推理框架，该框架使用由标准宽视场荧光显微镜在样本体积内的任意轴向位置稀疏捕获的二维图像。通过循环卷积神经网络（我们称之为 Recurrent-MZ），来自样品内几个轴向平面的 2D 荧光信息被明确合并，以在扩展的景深上以数字方式重建样品体积。通过对秀丽隐杆线虫和纳米珠样品的实验，证明 Recurrent-MZ 可以显着增加 63×/1.4NA 物镜的景深，同时将成像所需的轴向扫描次数减少 30 倍相同的样品体积。我们通过展示其对不同成像条件（包括不同的输入图像序列、涵盖各种轴​​向排列和未知的轴向定位误差）的弹性，进一步说明了该循环网络在 3D 成像中的泛化。我们还使用 Recurrent-MZ 框架演示了宽视场到共焦跨模态图像转换，并使用一些宽视场 2D 荧光图像作为输入对样品进行 3D 图像重建，匹配相同样品体积的共焦显微镜图像。 Recurrent-MZ 展示了循环神经网络在显微图像重建中的首次应用，并提供了灵活快速的体积成像框架，克服了当前 3D 扫描显微镜工具的局限性。