Object detection networks are high-performance algorithms famously applied to the task of identifying and localizing objects in photography images. We demonstrate their application for the classification and localization of cells in fluorescence microscopy by benchmarking four leading object detection algorithms across multiple challenging 2D microscopy datasets. Furthermore we develop and demonstrate an algorithm that can localize and image cells in 3D, in close to real time, at the microscope using widely available and inexpensive hardware. Furthermore, we exploit the fast processing of these networks and develop a simple and effective augmented reality (AR) system for fluorescence microscopy systems using a display screen and back-projection onto the eyepiece. We show that it is possible to achieve very high classification accuracy using datasets with as few as 26 images present. Using our approach, it is possible for relatively nonskilled users to automate detection of cell classes with a variety of appearances and enable new avenues for automation of fluorescence microscopy acquisition pipelines.

中文翻译：

---

用于荧光显微镜中细胞检测的物体检测网络和增强现实


对象检测网络是高性能算法，著名地应用于识别和定位摄影图像中的对象的任务。我们通过在多个具有挑战性的 2D 显微镜数据集中对四种领先的目标检测算法进行基准测试，展示了它们在荧光显微镜中细胞分类和定位的应用。此外，我们开发并演示了一种算法，可以使用广泛可用且廉价的硬件在显微镜上近乎实时地对细胞进行 3D 定位和成像。此外，我们利用这些网络的快速处理能力，开发了一种简单有效的增强现实（AR）系统，用于使用显示屏和目镜背投的荧光显微镜系统。我们证明，使用仅包含 26 个图像的数据集就可以实现非常高的分类精度。使用我们的方法，相对不熟练的用户可以自动检测具有各种外观的细胞类别，并为荧光显微镜采集管道的自动化提供新途径。