The motivation of our work is to present a new visualization-guided computing paradigm to combine direct 3D volume processing and volume rendered clues for effective 3D exploration such as extracting and visualizing microstructures in-vivo. However, it is still challenging to extract and visualize high fidelity 3D vessel structure due to its high sparseness, noisiness, and complex topology variations. In this paper, we present an end-to-end deep learning method, VC-Net, for robust extraction of 3D microvasculature through embedding the image composition, generated by maximum intensity projection (MIP), into 3D volume image learning to enhance the performance. The core novelty is to automatically leverage the volume visualization technique (MIP) to enhance the 3D data exploration at deep learning level. The MIP embedding features can enhance the local vessel signal and are adaptive to the geometric variability and scalability of vessels, which is crucial in microvascular tracking. A multi-stream convolutional neural network is proposed to learn the 3D volume and 2D MIP features respectively and then explore their inter-dependencies in a joint volume-composition embedding space by unprojecting the MIP features into 3D volume embedding space. The proposed framework can better capture small / micro vessels and improve vessel connectivity. To our knowledge, this is the first deep learning framework to construct a joint convolutional embedding space, where the computed vessel probabilities from volume rendering based 2D projection and 3D volume can be explored and integrated synergistically. Experimental results are compared with the traditional 3D vessel segmentation methods and the deep learning state-of-the-art on public and real patient (micro-)cerebrovascular image datasets. Our method demonstrates the potential in a powerful MR arteriogram and venogram diagnosis of vascular diseases.

中文翻译：

---

VC-Net：用于高度稀疏和噪声图像数据的分割和可视化的深度体积合成网络

我们工作的动机是提出一种新的可视化引导计算范式，将直接 3D 体积处理和体积渲染线索相结合，以进行有效的 3D 探索，例如在体内提取和可视化微结构。然而，由于其高稀疏性、噪声和复杂的拓扑变化，提取和可视化高保真 3D 血管结构仍然具有挑战性。在本文中，我们提出了一种端到端的深度学习方法 VC-Net，通过将最大强度投影 (MIP) 生成的图像合成嵌入到 3D 体积图像学习中来增强 3D 微血管系统的鲁棒提取性能. 核心创新是自动利用体积可视化技术 (MIP) 来增强深度学习级别的 3D 数据探索。MIP 嵌入特征可以增强局部血管信号并适应血管的几何可变性和可扩展性，这在微血管跟踪中至关重要。提出了一种多流卷积神经网络来分别学习 3D 体积和 2D MIP 特征，然后通过将 MIP 特征反投影到 3D 体积嵌入空间来探索它们在联合体积合成嵌入空间中的相互依赖性。所提出的框架可以更好地捕获小型/微型血管并改善血管连通性。据我们所知，这是第一个构建联合卷积嵌入空间的深度学习框架，其中可以协同探索和集成基于体积渲染的 2D 投影和 3D 体积计算的血管概率。在公共和真实患者（微）脑血管图像数据集上将实验结果与传统的 3D 血管分割方法和最先进的深度学习技术进行比较。我们的方法证明了强大的 MR 动脉造影和静脉造影诊断血管疾病的潜力。