Real-time multi-emitter fitting is a key technology for advancing super-resolution localization microscopy (SRLM), especially when it is necessary to achieve dynamic imaging quality control and/or optimization of experimental conditions. However, with the increase of activation densities, the requirements in the computing resources would increase rapidly due to the complexity of the fitting algorithms, making it difficult to realize real-time multi-emitter fitting for emitter density more than 0.6 mol/µm² in large field of view (FOV), even after acceleration with the popular Graphics Processing Unit (GPU) computation. Here we adopt the task parallelism strategy in computer science to construct a Peripheral Component Interconnect Express (PCIe) based all-in-one heterogeneous computing platform (AIO-HCP), where the data between two major parallel computing hardware, Field Programmable Gate Array (FPGA) and GPU, are interacted directly and executed simultaneously. Using simulated and experimental data, we verify that AIO-HCP could achieve a data throughput of up to ∼ 1.561 GB/s between FPGA and GPU. With this new platform, we develop a multi-emitter fitting method, called AIO-STORM, under big data stream parallel scheduling. We show that AIO-STORM is capable of providing real-time image processing on raw images with 100 µm × 100 µm FOV, 10 ms exposure time and 5.5 mol/µm² structure density, without scarifying image quality. This study overcomes the data throughput limitation of heterogeneous devices, demonstrates the power of the PCIe-based heterogeneous computation platform, and offers opportunities for multi-scale stitching of super-resolution images.

中文翻译：

---

基于 PCIe 的 FPGA-GPU 异构计算用于超分辨率定位显微镜中的实时多发射器拟合

实时多发射器拟合是推进超分辨率定位显微镜 (SRLM) 的关键技术，尤其是在需要实现动态成像质量控制和/或实验条件优化时。然而，随着激活密度的增加，由于拟合算法的复杂性，对计算资源的需求会迅速增加，难以实现发射体密度大于0.6 mol/µm ² in的实时多发射体拟合。大视场 (FOV)，即使在使用流行的图形处理单元 (GPU) 计算进行加速后也是如此。在这里，我们采用计算机科学中的任务并行策略来构建一个基于ll - in-在异构计算平台( AIO - HCP ) 中，两个主要并行计算硬件现场可编程门阵列 (FPGA) 和 GPU 之间的数据直接交互并同时执行。使用模拟和实验数据，我们验证 AIO-HCP 在 FPGA 和 GPU 之间可以实现高达 ∼ 1.561 GB/s 的数据吞吐量。借助这个新平台，我们在大数据流并行调度下开发了一种称为 AIO-STORM 的多发射器拟合方法。^{我们展示了 AIO-STORM 能够对具有 100 µm × 100 µm FOV、10 ms 曝光时间和 5.5 mol/µm 2}的原始图像进行实时图像处理结构密度，不影响图像质量。本研究克服了异构设备的数据吞吐量限制，展示了基于 PCIe 的异构计算平台的强大功能，并为超分辨率图像的多尺度拼接提供了机会。