SIAM Journal on Scientific Computing, Volume 43, Issue 5, Page A3328-A3356, January 2021.
We introduce a new theoretical framework for deriving lower bounds on data movement in bilinear algorithms. Bilinear algorithms are a general representation of fast algorithms for bilinear functions, which include computation of matrix multiplication, convolution, and symmetric tensor contractions. A bilinear algorithm is described by three matrices. Our communication lower bounds are based on quantifying the minimal matrix ranks of matching subsets of columns of these matrices. This infrastructure yields new lower bounds for symmetric tensor contraction algorithms, which provide new qualitative insights. Tensor symmetry (invariance under permutation of modes) is common to many applications of tensor computations (e.g., tensor representation of hypergraphs, analysis of high-order moments in data, as well as tensors modeling interactions of electrons in computational chemistry). Tensor symmetry enables reduction in representation size as well as contraction cost by factors that scale with the number of equivalent permutations. However, we derive lower bounds showing that these cost and memory reductions can necessitate increases in data movement by factors that scale with the size of the tensors.

中文翻译：

---

对称张量收缩的双线性算法的通信下界

SIAM 科学计算杂志，第 43 卷，第 5 期，第 A3328-A3356 页，2021 年 1 月。
我们引入了一个新的理论框架，用于推导双线性算法中数据移动的下限。双线性算法是双线性函数快速算法的一般表示，包括矩阵乘法、卷积和对称张量收缩的计算。双线性算法由三个矩阵描述。我们的通信下限基于量化这些矩阵的列的匹配子集的最小矩阵等级。这种基础设施为对称张量收缩算法产生了新的下限，从而提供了新的定性见解。张量对称性（模式排列下的不变性）对于张量计算的许多应用（例如，超图的张量表示，数据中高阶矩的分析，以及模拟计算化学中电子相互作用的张量）。张量对称性可以通过与等效排列数量成比例的因子来减少表示大小和收缩成本。然而，我们得出的下限表明，这些成本和内存的减少可能会导致数据移动的增加，这些因素会随着张量的大小而变化。