Deep Learning (DL) models are becoming larger, because the increase in model size might offer significant accuracy gain. To enable the training of large deep networks, data parallelism and model parallelism are two well-known approaches for parallel training. However, data parallelism does not help reduce memory footprint per device. In this work, we introduce Large deep 3D ConvNets with Automated Model Parallelism (LAMP) and investigate the impact of both input's and deep 3D ConvNets' size on segmentation accuracy. Through automated model parallelism, it is feasible to train large deep 3D ConvNets with a large input patch, even the whole image. Extensive experiments demonstrate that, facilitated by the automated model parallelism, the segmentation accuracy can be improved through increasing model size and input context size, and large input yields significant inference speedup compared with sliding window of small patches in the inference. Code is available\footnote{https://monai.io/research/lamp-automated-model-parallelism}.

中文翻译：

---

LAMP：具有用于图像分割的自动模型并行性的大型深度网络

深度学习 (DL) 模型变得越来越大，因为模型大小的增加可能会显着提高精度。为了实现大型深度网络的训练，数据并行和模型并行是并行训练的两种众所周知的方法。但是，数据并行性无助于减少每个设备的内存占用。在这项工作中，我们引入了具有自动模型并行性 (LAMP) 的大型深度 3D ConvNets，并研究了输入和深度 3D ConvNets 的大小对分割精度的影响。通过自动模型并行，可以使用大输入块甚至整个图像来训练大型深度 3D ConvNets。大量实验表明，在自动模型并行性的推动下，可以通过增加模型大小和输入上下文大小来提高分割精度，与推理中小块的滑动窗口相比，大输入产生显着的推理加速。代码可用\footnote{https://monai.io/research/lamp-automated-model-parallelism}。