Distributed systems have been widely adopted for deep neural networks model training. However, the scalability of distributed training systems is largely bounded by the communication cost. We design a highly efficient collective communication library, namely Alibaba Collective Communication Library (ACCL), to build distributed training systems with linear scalability. ACCL provides optimized algorithms to fully make use of heterogeneous interconnects simultaneously. And the experimental results show significant performance improvement.

中文翻译：

---

ACCL：利用高效的集体通信库构建高度可扩展的分布式培训系统


分布式系统已广泛用于深度神经网络模型训练。然而，分布式训练系统的可扩展性很大程度上受到通信成本的限制。我们设计了一个高效的集体通信库，即阿里巴巴集体通信库（ACCL），以构建具有线性可扩展性的分布式训练系统。 ACCL 提供优化算法以同时充分利用异构互连。实验结果显示性能显着提升。