In a parallel computing scenario, the synchronization overhead, needed to coordinate the execution on the parallel computing nodes, can significantly impair the overall execution performance. Typically, synchronization is achieved by adopting a global synchronization schema involving all the nodes. In many application domains, though, a looser synchronization schema, namely, local synchronization, can be exploited, in which each node needs to synchronize only with a subset of the other nodes. In this work, we compare the performance of global and local synchronization using the efficiency, i.e., the ratio between the useful computing time and the total computing time, including the synchronization overhead, as a key performance indicator. We present an analytical study of the asymptotic behavior of the efficiency when the number of nodes increases. As an original contribution, we prove, using the Max-Plus algebra, that there is a non-zero lower bound on the efficiency in the case of local synchronization and we present a statistical procedure to find a value of this bound. This outcome marks a significant advantage of local synchronization with respect to global synchronization, for which the efficiency tends to zero when increasing the number of nodes.

中文翻译：

---

并行计算中全局和局部同步分析

在并行计算场景中，协调并行计算节点上的执行所需的同步开销会显着影响整体执行性能。通常，同步是通过采用涉及所有节点的全局同步模式来实现的。然而，在许多应用领域中，可以利用更松散的同步模式，即本地同步，其中每个节点只需要与其他节点的一个子集同步。在这项工作中，我们使用效率来比较全局和本地同步的性能，即有用计算时间与总计算时间之间的比率，包括同步开销，作为关键性能指标。我们对节点数量增加时效率的渐近行为进行了分析研究。作为原始贡献，我们使用 Max-Plus 代数证明，在本地同步的情况下，效率存在非零下限，并且我们提出了一个统计程序来找到该边界的值。这一结果标志着本地同步相对于全局同步的显着优势，当增加节点数量时效率趋于零。