New bounds on the field size for maximally recoverable codes instantiating grid-like topologies

Abstract

In recent years, the rapidly increasing amounts of data created and processed through the internet resulted in distributed storage systems employing erasure coding based schemes. Aiming to balance the tradeoff between data recovery for correlated failures and efficient encoding and decoding, distributed storage systems employing maximally recoverable codes came up. Unifying a number of topologies considered both in theory and practice, Gopalan et al. [15] initiated the study of maximally recoverable codes for grid-like topologies. In this paper, we focus on the maximally recoverable codes that instantiate grid-like topologies $T_{m\times n}(1,b,0)$. To characterize the property of codes for these topologies, we introduce the notion of pseudo-parity check matrix. Then, using the Combinatorial Nullstellensatz, we establish the new upper bound on the field size needed for achieving the maximal recoverability in topologies $T_{m\times n}(1,b,0)$. By relating the problem to generalized Sidon sets in ${\mathbb {F}}_q$, we obtain a polynomial lower bound on the field size for maximally recoverable codes that instantiate topologies $T_{m\times n}(1,2,0)$. Moreover, using hypergraph independent set approach, we further improve our general upper bound for topologies $T_{4\times n}(1,2,0)$ and $T_{3\times n}(1,3,0)$.

1 Introduction

With rapidly increasing amounts of data created and processed in internet scale companies such as Google, Facebook, and Amazon, the efficient storage of such copious amounts of data has thus become a fundamental and acute problem in modern computing. This resulted in distributed storage systems relying on distinct storage nodes. Modern large scale distributed storage systems, such as data centers, used to store data in a redundant form to ensure reliability against node failures. However, this strategy entails large storage overhead and is nonadaptive for modern systems supporting the “Big Data” environment.

To ensure the reliability with better storage efficiency, erasure coding schemes are employed, such as in Windows Azure [19] and in Facebook’s Hadoop cluster [34]. However, in traditional erasure coding scheme, if one node fails, which is the most common failure scenario, we may recover it by accessing a large amount of the remaining nodes. This is a time consuming recovery process. To address this efficiency problem, a lot of works have emerged in two aspects: local regeneration and local reconstruction.

The concept of local regeneration was introduced by Dimakis et al. [8]. They established a tradeoff between the repair bandwidth and the storage capacity of a node, and introduced a new family of codes, called regenerating codes, which attain this tradeoff. The concept of local reconstruction was introduced by Gopalan et al. [14], and they initiated the study of Local Reconstruction Codes (LRCs). We say a certain node has locality r if it can be recovered by accessing only r other nodes, and LRCs are linear codes with all-symbol locality r. In recent years, the theory of regenerating codes and LRCs has developed rapidly. There have been a lot of related works focusing on the bounds and the constructions of optimal codes, see [20, 24, 31,32,33, 35, 39,40,41,42, 44, 46, 47] and the reference therein.

The notion of maximally recoverable property was first introduced by Chen et al. [7] for multi-protection group codes, and then extended by Gopalan et al. [13] to general settings. In [13], the authors introduced the topology of the code to specify the supports for the parity check equations, and they also obtained a general upper bound on the minimal size of the field over which maximally recoverable (MR) codes exist.

Different from the parity check matrix, the topology of the code only specifies the number of redundant symbols and the data symbols on which the redundant ones depend. This makes the topology a crucial characterization of the structure of the code used under distributed storage settings. With the purpose of deploying longer codes in storage, Gopalan et al. [15] proposed a family of topologies called grid-like topologies, which unified a number of topologies considered both in theory and practice.

Consider an $m\times n$ matrix, each entry storing a data from a finite field ${\mathbb {F}}$. Every row satisfies a given set of a parity constraints, and every column satisfies a given set of b parity constraints. In addition, there are h global parity constraints that involve all mn entries from the matrix. The topology of the code under these three constraints is denoted by $T_{m\times n}(a,b,h)$. In [15], the authors considered the maximal recoverable codes for general grid-like topologies, and they established a super-polynomial lower bound on the field size needed for achieving maximal recoverability in any grid-like topologies $T_{m\times n}(a,b,h)$ with $a,b,h\ge 1$. They also tried to characterize correctable erasure patterns for grid-like topologies of the form $T_{m\times n}(a,b,0)$, and obtained a full combinatorial characterization for the case of $T_{m\times n}(1,b,0)$.

The general lower bound given in [15] is obtained from the case of a basic topology $T_{n\times n}(1,1,1)$, where the lower bound requires field size $q=2^{\Omega ((\log n)^2)}$. Recently, by relating the problem to the independence number of the Birkhoff polytope graph, Kane et al. [21] improved the lower bound to $q\ge 2^{(\frac{n}{2}-2)}$ using the representation theory of the symmetric group. They also obtained an upper bound $q\le 2^{3n}$ using recursive constructions.

As for other related works, Gandikota et al. [12] considered the maximal recoverability for erasure patterns of bounded size. Shivakrishna et al. [36] considered the recoverability of a special kind of erasure patterns called extended erasure patterns for topologies $T_{(m+m')\times n}(2,b,0)$. It is worth noting that, Gopi et al. [16] recently obtained a super-linear lower bound for maximally recoverable LRCs which can be viewed as the MR codes for topology $T_{\frac{n}{r}\times r}(a,0,h)$.

In this paper, we focus on the maximally recoverable codes that instantiate topologies of the form $T_{m\times n}(1,b,0)$, which can be regarded as tensor product codes of column codes with a single parity constraint and row codes with b parity constraints. In order to describe the parity constraints globally, we introduce the notion of pseudo-parity check matrix, which can be viewed as a generalization of the parity check matrix. Based on this, we prove the following results:

• A general upper bound on the minimal size of the field required for the existence of MR codes that instantiate topologies $T_{m\times n}(1,b,0)$:

  $$\begin{aligned} q\le (m-1)\cdot {m\cdot b(m-1)\atopwithdelims ()2b(m-1)}\cdot n^{bm-b}+n^{b-1}. \end{aligned}$$

• A polynomial lower bound on the minimal size of the field required for MR codes instantiating topologies $T_{m\times n}(1,2,0)$:

  $$\begin{aligned} q\ge \frac{(n-3)^{2}}{4}+2. \end{aligned}$$

  As far as we know, this is the first super linear lower bound on field size of MR tensor codes under any setting.

• Further improved upper bounds on the field size required for MR codes instantiating topologies $T_{4\times n}(1,2,0)$ and $T_{3\times n}(1,3,0)$:

  $$\begin{aligned} q\le {\mathcal {O}}(\frac{n^{5}}{\log (n)}). \end{aligned}$$

The paper is organised as follows: In Section II, we give the formal definitions for general topologies, grid-like topologies and maximal recoverability. We also include some known results for topologies $T_{m\times n}(a,b,0)$ and the tools from hypergraph independent set. In Section III, we introduce the notion of pseudo-parity check matrix and regular irreducible erasure patterns. In Section IV, we present our proof for the general upper bound on the minimal size of the field required for the existence of MR codes that instantiate topologies $T_{m\times n}(1,b,0)$. In Section V, we establish a polynomial lower bound for MR codes that instantiate topologies $T_{m\times n}(1,2,0)$, and we also improve the general upper bound for some special cases. In Section VI, we conclude our work and list some open problems.

2 Preliminaries

2.1 Notation

We use the following standard mathematical notations throughout this paper.

• Let q be the power of a prime p, ${\mathbb {F}}_q$ be the finite field with q elements, ${\mathbb {F}}_{q}^{n}$ be the vector space of dimension n over ${\mathbb {F}}_q$ and ${\mathbb {F}}_{q}^{m\times n}$ be the collection of all $m\times n$ matrices with elements in ${\mathbb {F}}_q$.

• For any vector $\mathbf{v} =(v_1,\cdots ,v_n)\in {\mathbb {F}}_{q}^{n}$, let $\mathrm{supp}(\mathbf{v} )=\{i\in [n] : v_i\ne 0\}$ and ${\omega }(\mathbf{v} )=|\mathrm{supp}(v)|.$ For a set $S\subseteq [n],$ define $\mathbf{v} |_S=(v_{i_1},\ldots ,v_{i_{|S|}})$, where $i_j\in S$ for $1\le j\le |S|$ and $1\le i_1<\cdots <i_{|S|}\le n$.

• $[n,k,d]_q$ denotes a linear code of length n, dimension k and distance d over the field ${\mathbb {F}}_q$. We will write [n, k, d] instead of $[n,k,d]_q$ when the particular choice of the field is not important.

• Let ${\mathcal {C}}$ be an [n, k, d] code and $S\subseteq [n]$, $|S|=k$. We say that S is an information set if the restriction ${\mathcal {C}}|_{S}={\mathbb {F}}_q^k$.

• An [n, k, d] code is called Maximum Distance Separable (MDS) if $d=n-k+1$. Particularly, an [n, k, d] code is MDS if and only if every subset of its k coordinates is an information set. Alternatively, an [n, k, d] code is MDS if and only if it corrects any collection of $(n-k)$ simultaneous erasures (see [25]).

• Let ${\mathcal {C}}_1$ be an $[n_1,k_1,d_1]$ code and ${\mathcal {C}}_2$ be an $[n_2,k_2,d_2]$ code. The tensor product ${\mathcal {C}}_1\otimes {\mathcal {C}}_2$ is an $[n_1n_2,k_1k_2,d_1d_2]$ code such that the codewords of ${\mathcal {C}}_1\otimes {\mathcal {C}}_2$ are matrices of size $n_1\times n_2$, where each column belongs to ${\mathcal {C}}_1$ and each row belongs to ${\mathcal {C}}_2$. If $U\subseteq [n_1]$ is an information set of ${\mathcal {C}}_1$ and $V\subseteq [n_2]$ is an information set of ${\mathcal {C}}_2$, then $U\times V$ is an information set of ${\mathcal {C}}_1\otimes {\mathcal {C}}_2$ (see [25]).

• Let ${\mathbf {I}}_n$ be the $n\times n$ identity matrix. And let $\mathbf{1} _n$ and $\mathbf{0} _{n}$ be the all-one and all-zero vectors, respectively.

2.2 Maximal recoverability for general topologies

Let $z_1,\ldots ,z_m$ be variables over the field ${\mathbb {F}}_q$. Consider an $(n-k)\times n$ matrix $P=\{p_{ij}\}$ where each $p_{ij}\in {\mathbb {F}}_p[z_1,\ldots ,z_m]$ is an affine function of the $z_i$s over ${\mathbb {F}}_p$:

$$\begin{aligned} p_{ij}(z_1,\ldots ,z_m)=c_{ij0}+\sum \limits _{k=1}^m c_{ijk}z_k,~~~~c_{ijk}\in {\mathbb {F}}_p. \end{aligned}$$

(1)

We refer the matrix P as a topology. Fix an assignment $\{z_i=\alpha _i\}_{i=1}^{m}$, where $\alpha _i\in {\mathbb {F}}_q$. Viewing $P(\alpha _1,\ldots ,\alpha _m)$ as a parity check matrix, then it defines a linear code which is denoted by ${\mathcal {C}}(\alpha _1,\ldots ,\alpha _m)$. And we say code ${\mathcal {C}}$ instantiates P. Intuitively, the topology defined above characterizes the structure of the linear dependency among all the coordinates and represents a family of parity check matrices sharing same structure. For example, let $m=n$, take $c_{ijj}=a_j^{i-1}$ for n distinct $a_j\in {\mathbb {F}}_q\setminus \{0\}$ and $c_{ijk}=0$ for all the other $(i,j,k)\in [n-k]\times [n]\times ([n]\cup \{0\})$. Then for an assignment $\{z_i=\alpha _i\}_{i=1}^{m}\subseteq {\mathbb {F}}_q\setminus \{0\}$, $p_{ij}=\alpha _ja_j^{i-1}$ and $P(\alpha _1,\ldots ,\alpha _m)$ is a parity check matrix of an [n, k]-RS code.

A set $S\subseteq [n]$ of columns of P is called potentially independent if there exists an assignment $\{z_i=\alpha _i\}_{i=1}^{m}$ where $\alpha _i\in {\mathbb {F}}_q$ such that the columns of $P(\alpha _1,\ldots ,\alpha _m)$ indexed by S are linearly independent.

Definition 2.1

[13] The code ${\mathcal {C}}(\alpha _1,\ldots ,\alpha _m)$ instantiating the topology P is called maximally recoverable if every set of columns that is potentially independent in P is linearly independent in $P(\alpha _1,\ldots ,\alpha _m)$.

Using the Sparse Zeros Lemma (see Theorem 6.13 in [23]), Gopalan et al. [13] proved the following upper bound on the size of field over which the maximally recoverable codes for any topologies P exist.

Theorem 2.2

[13] Let $P\in ({\mathbb {F}}_p[z_1,\ldots ,z_m])^{(n-k)\times n}$ be an arbitrary topology. If $q>(n-k)\cdot {n\atopwithdelims (){\le n-k}}$, then there exists an MR instantiation of P over the field ${\mathbb {F}}_q$.

2.3 Grid-like topologies

Unifying and generalizing a number of topologies considered both in coding theory and practice, Gopalan et al. [15] proposed the following family of topologies called grid-like topologies via dual constraints.

Definition 2.3

[15] Let $m\le n$ be integers. Consider an $m\times n$ array of symbols $\{x_{ij}\}_{i\in [m],j\in [n]}$ over the field ${\mathbb {F}}_q$. Let $0\le a\le m-1$, $0\le b\le n-1$, and $0\le h\le (m-a)(n-b)-1$. Let $T_{m\times n}(a,b,h)$ denote the topology where there are a parity check equations per column, b parity check equations per row, and h global parity check equations that depend on all symbols. Topologies of the form $T_{m\times n}(a,b,h)$ are called grid-like topologies.

Furthermore, we say a collection of arrays ${\mathcal {C}}$ in ${\mathbb {F}}_q^{m\times n}$ to be a code that instantiates the topology $T_{m\times n}(a,b,h)$, if there exist $\{\alpha _i^{(k)}\}_{i\in [m],k\in [a]}$, $\{\beta _j^{(k)}\}_{j\in [n],k\in [b]}$ and $\{\gamma _{ij}^{(k)}\}_{i\in [m],j\in [n],k\in [h]}$ in ${\mathbb {F}}_q$ such that for each codeword $C=(c_{ij})_{i\in [m],j\in [n]}\in {\mathcal {C}}$:

• 1. Each column $j\in [n]$ satisfies the constraints

  $$\begin{aligned} \sum _{i=1}^{m}\alpha _{i}^{(k)}c_{ij}=0,~~\forall k\in [a]. \end{aligned}$$

  (2)

• 2. Each row $i\in [m]$ satisfies the constraints

  $$\begin{aligned} \sum _{j=1}^{n}\beta _{j}^{(k)}c_{ij}=0,~~\forall k\in [b]. \end{aligned}$$

  (3)

• 3. All the symbols satisfy h global constraints

  $$\begin{aligned} \sum _{i=1}^{m}\sum _{j=1}^{n}\gamma _{ij}^{(k)}c_{ij}=0,~~\forall k\in [h]. \end{aligned}$$

  (4)

Definition 2.4

An erasure pattern is a set $E\subseteq [m]\times [n]$ of symbols. Pattern E is correctable for the topology $T_{m\times n}(a,b,h)$ if there exists a code instantiating the topology where the variables $\{x_{ij}\}_{(i,j)\in E}$ can be recovered from the parity check equations (2), (3) and (4).

Clearly, constraints in (2) and (3) guarantee the local dependencies in each column and row respectively, and constraints in (4) ensure some additional recoverability. Notably, constraints in (2) specify a code ${\mathcal {C}}_{col}\subseteq {\mathbb {F}}_q^{m}$ and constraints in (3) specify a code ${\mathcal {C}}_{row}\subseteq {\mathbb {F}}_q^n$. If $h=0$, i.e., there are no extra global constraints for all symbols, then the code specified with the settings from Definition 2.3 is exactly the tensor product code ${\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$.

Definition 2.5

A code ${\mathcal {C}}$ that instantiates the topology $T_{m\times n}(a,b,h)$ is Maximally Recoverable (MR) if it can correct every erasure pattern that is correctable for the topology.

The maximally recoverability requires a code that instantiates the topology $T_{m\times n}(a,b,h)$ to have many good properties, especially the MDS property.

Proposition 2.6

[15] Let ${\mathcal {C}}$ be an MR instantiation of the topology $T_{m\times n}(a,b,h)$. We have

• 1. The dimension of ${\mathcal {C}}$ is given by

  $$\begin{aligned} dim~{\mathcal {C}}=(m-a)(n-b)-h. \end{aligned}$$

  (5)

  Moreover,

  $$\begin{aligned} dim~{\mathcal {C}}_{col}=m-a~~and~~dim~{\mathcal {C}}_{row}=n-b. \end{aligned}$$

  (6)

• 2. Let $U\subseteq [m]$, $|U|=m-a$ and $V\subseteq [n]$, $|V|=n-b$ be arbitrary. Then ${\mathcal {C}}|_{U\times V}$ is an

  $$\begin{aligned}{}[(m-a)(n-b),~(m-a)(n-b)-h,~h+1] \end{aligned}$$

  MDS code. Any subset $S\subseteq U\times V$, $|S|=(m-a)(n-b)-h$ is an information set.

• 3. Assume

  $$\begin{aligned} h\le (m-a)(n-b)-\max {\{(m-a),(n-b)\}}, \end{aligned}$$

  (7)

  then the code ${\mathcal {C}}_{col}$ is an $[m,m-a,a+1]$ MDS code and the code ${\mathcal {C}}_{row}$ is an $[n,n-b,b+1]$ MDS code. Moreover, for all $j\in [n]$, ${\mathcal {C}}$ restricted to column j is the code ${\mathcal {C}}_{col}$, and for all $i\in [m]$, ${\mathcal {C}}$ restricted to row i is the code ${\mathcal {C}}_{row}$.

Considering the topology $T_{m\times n}(a,b,0)$, the MR code ${\mathcal {C}}$ that instantiates this topology can be viewed as the tensor product code ${\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$. Based on the MDS properties for both ${\mathcal {C}}_{col}$ and ${\mathcal {C}}_{row}$, for a corresponding erasure pattern, we know that if some column has less than $a+1$ erasures or some row has less than $b+1$ erasures, we can decode it. Therefore, the erasure pattern that really matters shall have at least $a+1$ erasures in each column and at least $b+1$ erasures in each row.

Definition 2.7

An erasure pattern $E\subseteq [m]\times [n]$ for the topology $T_{m\times n}(a,b,0)$ is called irreducible, if for any $(i,j)\in E$, $|I(j)|=|\{i'\in [m]: (i',j)\in E\}|\ge a+1$ and $|J(i)|=|\{j'\in [n]: (i,j')\in E\}|\ge b+1$.

These kinds of patterns were originally mentioned in [15] and also appeared in [36]. While Gopalan et al. [15] were trying to characterize the correctable erasure patterns for grid-like topologies, they considered the natural question: are irreducible patterns uncorrectable? In order to address this question, they introduced the following notion of regularity for erasure patterns.

Definition 2.8

[15] Consider the topology $T_{m\times n}(a,b,0)$ and an erasure pattern E. We say that E is regular if for all $U\subseteq [m]$, $|U|=u$ and $V\subseteq [n]$, $|V|=v$ we have

$$\begin{aligned} |E\cap (U\times V)|\le va+ub-ab. \end{aligned}$$

(8)

By reducing the regular erasure patterns to the irreducible case, the authors proved the following equivalent condition of the correctable erasure patterns for the topology $T_{m\times n}(1,b,0)$.

Theorem 2.9

[15] An erasure pattern E is correctable for the topology $T_{m\times n}(1,b,0)$ if and only if it is regular for $T_{m\times n}(1,b,0)$.

2.4 Independent sets in hypergraphs

A hypergraph is a pair $(V,{\mathfrak {E}})$, where V is a finite set and ${\mathfrak {E}}\subseteq 2^{V}$ is a family of subsets of V. The elements of V are called vertices and the subsets in ${\mathfrak {E}}$ are called hyperedges. An independent set of a hypergraph is a set of vertices containing no hyperedges and the independence number of a hypergraph is the size of its largest independent set.

There are many results on the independence number of hypergraphs obtained through different methods (see [2, 3, 9, 22]). In the following section, we will apply the general lower bound derived by Kostochka et al. [22]. Before stating their theorem, we need a few definitions and notations. Let $H(V,{\mathfrak {E}})$ be a hypergraph with vertex set V and hyperedge set ${\mathfrak {E}}$. We call H a k-uniform hypergraph, if all the hyperedges have the same size k, i.e., ${\mathfrak {E}}\subseteq {V\atopwithdelims ()k}$. For any vertex $v\in V$, we define the degree of v to be the number of hyperedges containing v, denoted by d(v). The maximum of the degrees of all the vertices is called the maximum degree of H and denoted by $\Delta (H)$. The independence number of H is denoted by $\alpha (H)$. For a set R of r vertices, define the r-degree of R to be the number of hyperedges containing R.

Theorem 2.10

[22] Fix $r\ge 2$. There exists $c_r>0$ such that if H is an $(r+1)$-graph on n vertices with maximum r-degree $\Delta _r<n/(\log n)^{3r^2}$, then

$$\begin{aligned} \alpha (H)\ge c_r(\frac{n}{\Delta _r}\log {\frac{n}{\Delta _r}})^{\frac{1}{r}}, \end{aligned}$$

(9)

where $c_r>0$ and $c_r\sim r/e$ as $r\rightarrow \infty$.

3 Pseudo-parity check matrix and Regular irreducible erasure patterns

In this section, we shall introduce two important notions: pseudo-parity check matrix and regular irreducible erasure patterns, which are crucial in the proofs of both upper and lower bounds.

3.1 Pseudo-parity check matrix

Let ${\mathcal {C}}$ be an [n, k] linear code with a parity check matrix ${\mathbf {H}}\in {\mathbb {F}}_q^{(n-k)\times n}$, then we have the following well-known fact about ${\mathbf {H}}$.

Fact 3.1

[25] Assume a subset $E\subseteq [n]$ of the coordinates of ${\mathcal {C}}$ are erased, then they can be recovered if and only if the parity check matrix ${\mathbf {H}}$ restricted to coordinates in E has full rank.

Take ${\mathcal {C}}={\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$ as the tensor product code that instantiates the topology $T_{m\times n}(a,b,0)$, where ${\mathcal {C}}_{col}$ and ${\mathcal {C}}_{row}$ are codes specified by (2) and (3), respectively. For simplicity, for each codeword $c\in {\mathcal {C}}$ write

$$\begin{aligned} c=(c_{11},\ldots ,c_{1n},c_{21},\ldots ,c_{2n},\ldots ,c_{m1},\ldots ,c_{mn}), \end{aligned}$$

where for each $j\in [n], (c_{1j},\ldots ,c_{mj})$ is a codeword in ${\mathcal {C}}_{col}$ and for each $i\in [m], (c_{i1},\ldots ,c_{in})$ is a codeword in ${\mathcal {C}}_{row}$.

Denote ${\mathbf {H}}_{col}$ and ${\mathbf {H}}_{row}$ as the parity check matrices of ${\mathcal {C}}_{col}$ and ${\mathcal {C}}_{row}$ respectively, assume

$$\begin{aligned} {\mathbf {H}}_{col}=\left( \begin{array}{cccc} \alpha _1^{(1)} &{} \alpha _2^{(1)} &{}\cdots &{} \alpha _m^{(1)} \\ \alpha _1^{(2)} &{} \alpha _2^{(2)} &{} \ldots &{} \alpha _m^{(2)} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ \alpha _1^{(a)} &{} \alpha _2^{(a)} &{} \ldots &{} \alpha _m^{(a)} \\ \end{array} \right) ~\text {and}~ {\mathbf {H}}_{row}=\left( \begin{array}{cccc} \beta _1^{(1)} &{} \beta _2^{(1)} &{}\cdots &{} \beta _n^{(1)} \\ \beta _1^{(2)} &{} \beta _2^{(2)} &{} \ldots &{} \beta _n^{(2)} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ \beta _1^{(b)} &{} \beta _2^{(b)} &{} \ldots &{} \beta _n^{(b)} \\ \end{array} \right) . \end{aligned}$$

Then consider the following $(an +bm )\times mn$ matrix:

$$\begin{aligned} {\mathbf {H}}_{(a,b,0)}=\left( \begin{array}{cccc} {\mathbf {H}}_1 &{} {\mathbf {H}}_2 &{}\cdots &{} {\mathbf {H}}_m \\ {\mathbf {H}}_{row} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {H}}_{row} &{} \ldots &{} {\mathbf {0}} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ {\mathbf {0}} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {H}}_{row} \\ \end{array} \right) , \end{aligned}$$

(10)

where

$$\begin{aligned} {\mathbf {H}}_{i}=\left( \begin{array}{ccccc} \mathbf {\alpha }_i &{} {\mathbf {0}} &{} {\mathbf {0}} &{}\cdots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} \mathbf {\alpha }_i &{} {\mathbf {0}} &{}\ldots &{} {\mathbf {0}} \\ \vdots &{} \vdots &{} \vdots &{}\ddots &{} \vdots \\ {\mathbf {0}} &{} {\mathbf {0}} &{} {\mathbf {0}} &{} \ldots &{} \mathbf {\alpha }_i \\ \end{array} \right) ~\hbox { is a matrix of size}\ (a\cdot n)\times n~\text {with}~ \mathbf {\alpha }_i=\left( \begin{array}{c} \alpha _i^{(1)}\\ \alpha _i^{(2)}\\ \vdots \\ \alpha _i^{(a)}\\ \end{array} \right) . \end{aligned}$$

(11)

From the above construction, we can see that ${\mathbf {H}}_{(a,b,0)}$ includes all the parity check constraints of ${\mathcal {C}}$, and it can be easily verified that ${\mathbf {H}}_{(a,b,0)}\cdot c^{T}=0$ for each codeword $c\in {\mathcal {C}}$. Since the size of ${\mathbf {H}}_{(a,b,0)}$ is $(an +bm )\times mn$, instead of the parity check matrix of ${\mathcal {C}}$, it can only be regarded as an approximation of the parity check matrix. Therefore, we call ${\mathbf {H}}_{(a,b,0)}$ a pseudo-parity check matrix of the code ${\mathcal {C}}$.

Similar to Fact 3.1, using basic linear algebra arguments, we have the following proposition for pseudo-parity check matrix of code ${\mathcal {C}}$.

Proposition 3.2

Assume a subset $E\subseteq [mn]$ of the coordinates of ${\mathcal {C}}$ are erased, then they can be recovered if and only if the pseudo-parity check matrix ${\mathbf {H}}_{(a,b,0)}$ restricted to coordinates in E has full column rank.

When $a=1$, ${\mathbf {H}}_{col}$ has rank 1. Especially, when considering the existence of MR codes for topologies $T_{m\times n}(1,b,0)$, w.l.o.g, we can fix ${\mathcal {C}}_{col}$ to be the simple parity code ${\mathcal {P}}_m$, i.e., ${\mathbf {H}}_{col}=(1~1~\cdots ~1)$. Hence, the pseudo-parity check matrix ${\mathbf {H}}_{(1,b,0)}$ of ${\mathcal {C}}={\mathcal {P}}_m\otimes {\mathcal {C}}_{row}$ has the form:

(12)

Remark 3.3

Let r|n and $g=\frac{n}{r}$, an (n, r, h, a, q)-MR LRC can be viewed as an MR code for topology $T_{g\times r}(a,0,h)$. Therefore, it has simpler erasure patterns compared to the tensor product cases. (Briefly speaking, an (n, r, h, a, q)-MR LRC is an $[n,n-ga-h]$ linear code with (r, a)-locality which can correct any erasure pattern E consisting of a erasures from each local group and any other h more erasures. For specific definition, please see [16].) And instead of using the pseudo-parity check matrix, it can be verified that the parity check matrix of any (n, r, h, a, q)-MR LRC admits the form

$$\begin{aligned} {\mathbf {H}}=\left( \begin{array}{cccc} {\mathbf {A}}_1 &{} {\mathbf {A}}_2 &{}\cdots &{} {\mathbf {A}}_{g} \\ {\mathbf {H}}_{1} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {H}}_{2} &{} \ldots &{} {\mathbf {0}} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ {\mathbf {0}} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {H}}_{g} \\ \end{array} \right) , \end{aligned}$$

where for each $i\in [g]$, ${\mathbf {H}}_i$ is a parity check matrix of an $[r,r-a,a+1]$ MDS code and ${\mathbf {A}}_i$ is an $h\times r$ matrix over ${\mathbb {F}}_q$ corresponding to the global parities.

Compared to MR LRCs, MR codes for topologies $T_{m\times n}(a,b,0)$ have another difference. For an (n, r, h, a, q)-MR LRC, the $[r,r-a,a+1]$ MDS codes within each local group can be different, this results in that the corresponding parity check matrix ${\mathbf {H}}$ above can admit different ${\mathbf {H}}_i$s. However, since an MR code for topology $T_{m\times n}(a,b,0)$ is actually a tensor product code ${\mathcal {C}}={\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$. Thus, for each $i\in [m]$, if we take coordinates in $\{n(i-1)+1,\ldots ,ni\}$ as a local group, once the code ${\mathcal {C}}_{row}$ is fixed, the corresponding $[n,n-b,b+1]$ MDS codes within each local group are all ${\mathcal {C}}_{row}$ and the corresponding parity check matrices in ${\mathbf {H}}_{(a,b,0)}$ are all ${\mathbf {H}}_{row}$.

3.2 Regular irreducible erasure patterns

Let $E\in [m]\times [n]$ be an erasure pattern of the topology $T_{m\times n}(a,b,0)$, then it can be presented in the following form:

$$\begin{aligned} E=\left( \begin{array}{ccccccc} * &{} * &{} * &{} * &{} \cdots &{} \circ &{} \circ \\ * &{} * &{} \circ &{} * &{} \cdots &{} \circ &{} *\\ \circ &{} * &{} \circ &{} * &{} \cdots &{} * &{} \circ \\ \vdots &{} \vdots &{} \vdots &{} \vdots &{} \ddots &{} \vdots &{} \vdots \\ \circ &{} \circ &{} \circ &{} * &{} \cdots &{} * &{} *\\ \end{array} \right) , \end{aligned}$$

where $*$ stands for the erasure and $\circ$ stands for the non-erasure. Give two different erasure patterns $E_1$ and $E_2$, we say that $E_1$ and $E_2$ are of the same type, if $E_2$ can be obtained from $E_1$ by applying elementary row and column transformations.

For a reducible erasure pattern E, there exists some $i_0\in [m]$ or $j_0\in [n]$, such that the number of the erasures in $E\cap [i_0]\times [n]$ or $E\cap [m]\times [j_0]$ is less than $b+1$ or $a+1$. Therefore, from the MDS properties of the code ${\mathcal {C}}_{row}$ and ${\mathcal {C}}_{col}$, erasures in $E\cap [i_0]\times [n]$ or $E\cap [m]\times [j_0]$ can be simply repaired by using only the parities within ${\mathbf {H}}_{row}$ or ${\mathbf {H}}_{col}$. Hence, the very erasure patterns that affect the MR property of the code ${\mathcal {C}}$ are irreducible erasure patterns. In other words, if we can construct a code ${\mathcal {C}}$ instantiating the topology $T_{m\times n}(a,b,0)$ that can correct all correctable irreducible erasure patterns, then this code ${\mathcal {C}}$ is an MR instantiation for the topology $T_{m\times n}(a,b,0)$.

Now, we focus on the irreducible erasure patterns that are correctable. Given an irreducible erasure pattern E, denote |E| as the number of $*$s in E, $U_E=\{i\in [m]: \exists j\in [n]~\text {such~that} ~E(i,j)=*\}$ and $V_E=\{j\in [n]: \exists i\in [m]~\text {such that}~ E(i,j)=*\}$. From the irreducibility of E, we have

$$\begin{aligned} |E|\ge (a+1)|V_E|~and~|E|\ge (b+1)|U_E|. \end{aligned}$$

Meanwhile, from Theorem 2.9, we know that for topology $T_{m\times n}(1,b,0)$, an erasure pattern E is correctable if and only if E is regular. Thus we have

$$\begin{aligned} |E|=|E\cap (U_E\times V_E)|\le a|V_E|+b|U_E|-ab=|V_E|+b|U_E|-b. \end{aligned}$$

Combining the above three inequalities together, we have

$$\begin{aligned} |U_E|+b\le |V_E|\le b|U_E|-b, \end{aligned}$$

(13)

for every correctable irreducible erasure patterns E in $T_{m\times n}(1,b,0)$. Therefore,

$$\begin{aligned} \max \{{2(|U_E|+b),(b+1)|U_E|}\}\le |E|\le 2b(|U_E|-1), \end{aligned}$$

(14)

which indicates that once $|U_E|$ (or $|V_E|$) is given, the magnitude of |E| can not be too large.

Denote ${\mathcal {E}}$ as the set of all the types of regular irreducible erasure patterns for topology $T_{m\times n}(1,b,0)$, i.e., for each $E\in {\mathcal {E}}$, one can regard E as a representative of all the erasure patterns that have the same type as E. Since $U_E\subseteq [m]$ and $V_E\subseteq [n]$, for each regular irreducible erasure pattern E, (13) and (14) show that $|V_E|\le b(m-1)$ and $|E|\le 2b(m-1)$. For convenience, we can take each type of erasure patterns in ${\mathcal {E}}$ as a submatrix of an $m\times b(m-1)$ matrix with elements from $\{*,\circ \}$. Therefore, we can obtain the following upper bound of $|{\mathcal {E}}|$:

$$\begin{aligned} |{\mathcal {E}}|\le {m\cdot b(m-1)\atopwithdelims ()\le 2b(m-1)}. \end{aligned}$$

(15)

For general topology $T_{m\times n}(1,b,0)$, due to complexity for examining regularity, a fully characterization of all types of regular irreducible erasure patterns can be very difficult. However, the following proposition shows that joints of erasure patterns maintain the regularity and the irreducibility, which might be useful for finding large regular irreducible erasure patterns.

Proposition 3.4

Let $E_1$ and $E_2$ be two regular irreducible erasure patterns for $T_{m\times n}(a,b,0)$. If $m\ge |U_{E_1}|+|U_{E_2}|$ and $n\ge |V_{E_1}|+|V_{E_2}|$, then the erasure pattern of the form

$$\begin{aligned} E'=\left( \begin{array}{cc} E_1 &{} \circ \\ \circ &{} E_2 \\ \end{array} \right) , \end{aligned}$$

is also regular irreducible for $T_{m\times n}(a,b,0)$.

Proof

The irreducibility of $E'$ follows easily from those of $E_1$ and $E_2$, thus we only have to consider the regularity.

Now, consider the index set $U\times V\subseteq [m]\times [n]$. If $U\setminus U_{E'}\ne \emptyset$ or $V\setminus V_{E'}\ne \emptyset$, then we can delete the corresponding rows and columns outside of $U_{E'}$ and $V_{E'}$, this leads to a smaller index set with the same number of erasures. Therefore, w.l.o.g., assume that $U\times V\subseteq U_{E'}\times V_{E'}=(U_{E_1}\sqcup U_{E_2})\times (V_{E_1}\sqcup V_{E_2})$, where $\sqcup$ means the union of two disjoint sets. Then, we have the following partition of $U\times V$:

$$\begin{aligned} U\times V= & {} (U\cap U_{E_1})\times (V\cap V_{E_1})\sqcup (U\cap U_{E_1})\times (V\cap V_{E_2})\sqcup (U\cap U_{E_2})\\&\times (V\cap V_{E_1})\sqcup (U\cap U_{E_2})\times (V\cap V_{E_2}). \end{aligned}$$

Thus

$$\begin{aligned} |E'\cap (U\times V)|=\sum _{(i,j)\in [2]\times [2]}|E_{(i,j)}|, \end{aligned}$$

where $E_{(i,j)}=E'\cap (U\cap U_{E_i})\times (V\cap V_{E_j})$. By the form of $E'$, we have $|E_{(1,2)}|=|E_{(2,1)}|=0$. Since the regularity of $E_1$ and $E_2$ implies that $|E_{(1,1)}|\le a|V\cap V_{E_1}|+b|U\cap U_{E_1}|-ab$ and $|E_{(2,2)}|\le a|V\cap V_{E_2}|+b|U\cap U_{E_2}|-ab$. Therefore, combining these two inequalities with the former identities, we have

$$\begin{aligned} |E'\cap (U\times V)|&\le (a|V\cap V_{E_1}|+a|V\cap V_{E_2}|)+b(|U\cap U_{E_1}|+|U\cap U_{E_2}|)-2ab \\&< a|V|+b|U|-ab. \end{aligned}$$

Thus, $E'$ is also a regular erasure pattern. $\square$

4 An upper bound on the minimal field size required for MR codes

In this section, we take the prime $p=2$, which is the natural setting for distributed storage. And we will establish our upper bound on the minimal field size required for MR codes that instantiate the topology $T_{m\times n}(1,b,0)$.

Theorem 4.1

Let $m,b\ge 1$. Then for any $q\ge (m-1)\cdot {m\cdot b(m-1)\atopwithdelims ()2b(m-1)}\cdot n^{bm-b}+n^{b-1}$, there exists an MR code ${\mathcal {C}}$ that instantiates the topology $T_{m\times n}(1,b,0)$ over the field ${\mathbb {F}}_q$.

In order to do this, we will exhibit a column code ${\mathcal {C}}_{col}$ and a row code ${\mathcal {C}}_{row}$ over a relative small field, so that for every correctable irreducible erasure pattern E, the code ${\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$ can correct E. Thus the tensor product code ${\mathcal {C}}={\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$ is an MR code that instantiates the topology $T_{m\times n}(1,b,0)$. We need the following lemma known as the Combinatorial Nullstellensatz.

Lemma 4.2

(Combinatorial Nullstellensatz) [1] Let ${\mathbb {F}}$ be an arbitrary field, let $P\in {\mathbb {F}}[t_1,\ldots ,t_n]$ be a polynomial of degree d which contains a non-zero coefficient at $t_1^{d_1}\cdots t_n^{d_n}$ with $d_1+\cdots +d_n=d$, and let $S_1,\ldots ,S_n$ be subsets of ${\mathbb {F}}$ such that $|S_i|>d_i$ for all $1\le i\le n$. Then there exist $x_1\in S_1,\ldots ,x_n\in S_n$ such that $P(x_1,\ldots ,x_n)\ne 0$.

Proof of Theorem 4.1

Since the case $m=1$ is trivial, w.l.o.g., we assume $m\ge 2$. For simplicity, we fix ${\mathcal {C}}_{col}$ as the simple parity code ${\mathcal {P}}_m$ and focus on obtaining the code ${\mathcal {C}}_{row}$.

Denote ${\mathcal {E}}$ as the set of all the types of regular irreducible erasure patterns for topology $T_{m\times n}(1,b,0)$. Assume the parity check matrix of the code ${\mathcal {C}}_{row}$ is ${\mathbf {H}}_{row}$, then the pseudo-parity check matrix ${\mathbf {H}}$ is of the form in (12). Thus, our goal is to construct a $b\times n$ matrix ${\mathbf {H}}_{row}$ such that:

• (i) Every b distinct columns of ${\mathbf {H}}_{row}$ are linearly independent.

• (ii) For each regular irreducible erasure pattern E, the pseudo-parity check matrix ${\mathbf {H}}\in {\mathbb {F}}_q^{(n+bm)\times mn}$ of ${\mathcal {C}}$ satisfies: $rank({\mathbf {H}}|_E)=|E|$.

The requirement (i) is to guarantee that the code ${\mathcal {C}}_{row}$ is an $[n,n-b]$-MDS code and by Proposition 3.2, the requirement (ii) guarantees that ${\mathcal {C}}$ can correct all regular irreducible erasure patterns.

Given a regular irreducible erasure pattern $E\in [m]\times [n]$, w.l.o.g., assume $U_E=[u_0]\subseteq [m]$ and $V_E=[v_0]\subseteq [n]$, then E has the form

$$\begin{aligned} E=\left( \begin{array}{ccccccc} * &{} * &{} * &{} * &{} \cdots &{} \circ &{} \circ \\ * &{} * &{} * &{} \circ &{} \cdots &{} * &{} \circ \\ \circ &{} * &{} \circ &{} * &{} \cdots &{} * &{} \circ \\ \vdots &{} \vdots &{} \vdots &{} \vdots &{} \ddots &{} \vdots &{} \vdots \\ \circ &{} \circ &{} \circ &{} * &{} \cdots &{} * &{} *\\ \end{array} \right) =\left( \begin{array}{c} E_{1}\\ E_{2}\\ E_{3}\\ \vdots \\ E_{u_0}\\ \end{array} \right) , \end{aligned}$$

where $E_i$ represents the sub-erasure pattern of E over the $i_{th}$ row. Thus

$$\begin{aligned} {\mathbf {H}}|_E=\left( \begin{array}{cccc} {\mathbf {I}}_n|_{E_{1}} &{} {\mathbf {I}}_n|_{E_{2}} &{}\cdots &{} {\mathbf {I}}_n|_{E_{u_0}} \\ {\mathbf {H}}_{row}|_{E_{1}} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {H}}_{row}|_{E_{2}} &{} \ldots &{} {\mathbf {0}} \\ \vdots &{} \vdots &{} \ddots &{} \vdots \\ {\mathbf {0}} &{} {\mathbf {0}} &{} \ldots &{} {\mathbf {H}}_{row}|_{E_{u_0}} \\ \end{array} \right) = \left( \begin{array}{c} {\mathbf {H}}_1|_E\\ {\mathbf {H}}_2|_E\\ \end{array} \right) . \end{aligned}$$

Let $supp(E_i)=\{j\in [n]:E_i(j)=*\}$. Since $\bigcup _{i=1}^{u_0} supp(E_i)=[v_0]$, by applying elementary row and column transformations, we have

$$\begin{aligned} {\mathbf {H}}|_E=\left( \begin{array}{cc} {\mathbf {I}}_{v_0} &{} {\mathbf {0}}_{v_0\times (|E|-v_0)}\\ {\mathbf {A}}_{u_0b\times v_0} &{} {\mathbf {B}}_{u_0b\times (|E|-v_0)}\\ {\mathbf {0}}_{(n-v_0)\times (v_0)} &{} {\mathbf {0}}_{(n-v_0)\times (|E|-v_0)}\\ \end{array} \right) , \end{aligned}$$

(16)

where ${\mathbf {A}}$ consists of all the columns in ${\mathbf {H}}_2|_E$ corresponding to an ${\mathbf {I}}_{v_0}$ in ${\mathbf {H}}_1|_E$ and ${\mathbf {B}}$ consists of all the rest columns in ${\mathbf {H}}_2|_E$ by substituting columns of ${\mathbf {A}}$ with the same parts in ${\mathbf {H}}_1|_E$. Thus a non-zero element of ${\mathbf {A}}$ equals to some $h_{ij}$ in ${\mathbf {H}}_{row}$ and a non-zero element of ${\mathbf {B}}$ equals to $h_{ij}$ or $-h_{ij}$ for some $h_{ij}$ in ${\mathbf {H}}_{row}$. For example, take $U_E=V_E=\{1,2,3\}$, ${\mathbf {H}}_{row}=(\mathbf{h} _1,\ldots ,\mathbf{h} _n)$ and

$$\begin{aligned} E=\left( \begin{array}{ccc} * &{} * &{} \circ \\ * &{} \circ &{} *\\ \circ &{} \circ &{} *\\ \end{array} \right) , \end{aligned}$$

then

From the above simplified form of ${\mathbf {H}}|_E$ in (16), we have

$$\begin{aligned} rank({\mathbf {H}}|_E)&=rank({\mathbf {I}}_{v_0})+rank({\mathbf {B}}_{u_0b\times (|E|-v_0)}) \\&=v_0+rank({\mathbf {B}}_{u_0b\times (|E|-v_0)}). \end{aligned}$$

By Definition 2.8 and (14), we have $|E|\le v_0+u_0b-b$. Thus $rank({\mathbf {H}}|_E)=|E|$ if and only if there exists an $(|E|-v_0)\times (|E|-v_0)$ minor ${\mathbf {B}}'$ in ${\mathbf {B}}$ such that $det({\mathbf {B}}')\ne 0$.

Now, take

$$\begin{aligned} {\mathbf {H}}_{row} =\left( \begin{array}{ccccc} x_{11} &{} x_{12} &{} x_{13} &{} \cdots &{} x_{1n}\\ x_{21} &{} x_{22} &{} x_{23} &{} \cdots &{} x_{2n}\\ x_{31} &{} x_{32} &{} x_{33} &{} \cdots &{} x_{3n}\\ \vdots &{} \vdots &{} \vdots &{} \ddots &{} \vdots \\ x_{b1} &{} x_{b2} &{} x_{b3} &{} \cdots &{} x_{bn}\\ \end{array} \right) , \end{aligned}$$

where each $x_{ij}$ is a variable over ${\mathbb {F}}_q$. Therefore, our goal is to find a proper valuation of these $x_{ij}'s$ over ${\mathbb {F}}_q$ such that the resulting matrix ${\mathbf {H}}_{row}$ satisfies both requirement (i) and requirement (ii).

• For requirement (i)

For any $J=\{j_1,\ldots ,j_b\}\subseteq [n]$, let ${\mathbf {M}}_{J}$ be the $b\times b$ submatrix of ${\mathbf {H}}_{row}$ formed by the b columns indicated by J, i.e.,

$$\begin{aligned} {\mathbf {M}}_{J} =\left( \begin{array}{ccccc} x_{1j_1} &{} x_{1j_2} &{} x_{1j_3} &{} \cdots &{} x_{1j_b}\\ x_{2j_1} &{} x_{2j_2} &{} x_{2j_3} &{} \cdots &{} x_{2j_b}\\ x_{3j_1} &{} x_{3j_2} &{} x_{3j_3} &{} \cdots &{} x_{3j_b}\\ \vdots &{} \vdots &{} \vdots &{} \ddots &{} \vdots \\ x_{bj_1} &{} x_{bj_2} &{} x_{bj_3} &{} \cdots &{} x_{bj_b}\\ \end{array} \right) . \end{aligned}$$

Define

$$\begin{aligned} P=\prod _{J\in {[n]\atopwithdelims ()b}}det({\mathbf {M}}_{J}). \end{aligned}$$

Since each $det({\mathbf {M}}_{J})$ is a homogeneous polynomial of degree b, we know that P is a homogeneous polynomial of degree $b{n\atopwithdelims ()b}$, and each variable $x_{ij}$ has degree at most ${{n-1}\atopwithdelims (){b-1}}$. According to the definition of P, if there is a valuation $(h_{11},\ldots ,h_{bn})$ of $(x_{11},\ldots ,x_{bn})$ such that $P(h_{11},\ldots ,h_{bn})\ne 0$, then the resulting matrix ${\mathbf {H}}_{row}=(h_{ij})_{i\in [b],j\in [n]}$ satisfies requirement (i).

• For requirement (ii)

For each regular irreducible erasure pattern $E\in [m]\times [n]$, set $|U_E|=u_0$ and $|V_E|=v_0$ and consider the $u_0b\times (|E|-v_0)$ submatrix ${\mathbf {B}}(E)$ of ${\mathbf {H}}|_E$ in (16). For each $(|E|-v_0)\times (|E|-v_0)$ minor ${\mathbf {B}}'(E)$ in ${\mathbf {B}}(E)$, $det({\mathbf {B}}'(E))$ can be viewed as a multi-variable polynomial in ${\mathbb {F}}_q[x_{11},\ldots ,x_{bn}]$ with degree at most $|E|-v_0$. Since each non-zero element of ${\mathbf {B}}$ equals to $x_{ij}$ or $-x_{ij}$ for some $x_{ij}$ in ${\mathbf {H}}_{row}$, and each variable $x_{ij}$ appears in at most $u_0-1$ columns of ${\mathbf {B}}$, thus for each minor ${\mathbf {B}}'(E)$ we have

$$\begin{aligned} det({\mathbf {B}}'(E))=\sum _{\begin{array}{c} \sum _{1\le i\le b,1\le j\le n} a_{ij}=|E|-v_0,\\ 0\le a_{ij}\le u_0-1 \end{array}} c_{(a_{11},\ldots ,a_{bn})}\cdot x_{11}^{a_{11}}x_{12}^{a_{12}}\cdots x_{bn}^{a_{bn}}, \end{aligned}$$

(17)

where $c_{(a_{11},\ldots ,a_{bn})}$ equals to 0, 1 or $-1$.

Since elementary row and column transformations don’t change the form of ${\mathbf {H}}|_{E}$ (In fact, these transformations only affect the indices of ${\mathbf {h}}_i$s, and therefore, switch $x_{i,j}$ to $x_{i,j'}$ for $j'\ne j\in [n]$), the structure of ${\mathbf {B}}'(E)$ is determined by the type of the erasure pattern. Therefore, once E is given, for each minor ${\mathbf {B}}'(E)$ in ${\mathbf {B}}(E)$, $det({\mathbf {B}}'(E))$ can be viewed as a polynomial in ${\mathbb {F}}_2[x_{11},\ldots ,x_{bn}]$ with a fixed form.

Since for each regular irreducible erasure pattern E, $|E|\le v_0+bm-b$. Thus, when $q >bm-b\ge \deg (det({\mathbf {B}}'(E)))$, $det({\mathbf {B}}'(E))|_{{\mathbb {F}}_q^{(bn)}}\equiv 0$ if and only if $det({\mathbf {B}}'(E))={\mathbf {0}}$ (i.e. the zero polynomial).

According to the proof of Theorem 2.9 in [15], when the size of the field is large enough, there exists a code ${\mathcal {C}}_{0}$ such that the tensor product code ${\mathcal {C}}={\mathcal {P}}_m\otimes {\mathcal {C}}_{0}$ can correct E. This means that there exists a valuation of the bn variables in ${\mathbf {H}}_{row}$ such that $det({\mathbf {B}}'(E))\ne 0$ for some $(|E|-|V_E|)\times (|E|-|V_E|)$ minor ${\mathbf {B}}'(E)$ in ${\mathbf {B}}(E)$. By this, we know that the multi-variable polynomial $det({\mathbf {B}}'(E))$ corresponding to this minor ${\mathbf {B}}'(E)$ can not be zero polynomial. From the previous analysis, we know that the form of this polynomial $det({\mathbf {B}}'(E))$ is irrelevant to the size of the field. Therefore, for any $q> bm-b$ as a power of 2, this $det({\mathbf {B}}'(E))$ is a non-zero polynomial in ${\mathbb {F}}_q[x_{11},\ldots ,x_{bn}]$.

For each regular irreducible erasure pattern E, denote $f_E$ as the non-zero determinant polynomial corresponding to some $(|E|-|V_E|)\times (|E|-|V_E|)$ minor ${\mathbf {B}}'(E)$ in ${\mathbf {B}}(E)$. Define

$$\begin{aligned} F=\prod _{\begin{array}{c} E\in [m]\times [n], \\ E\text {~is a regular irreducible erasure pattern} \end{array}}f_E. \end{aligned}$$

(18)

Similarly, if there is a valuation $(h_{11},\ldots ,h_{bn})$ of $(x_{11},\ldots ,x_{bn})$ such that $F(h_{11},\ldots ,h_{bn})\ne 0$, then the resulting matrix ${\mathbf {H}}_{row}=(h_{ij})_{i\in [b],j\in [n]}$ satisfies requirement (ii).

In order to apply the Combinatorial Nullstellensatz, we shall estimate the degree of each variable in F. Noted that

$$\begin{aligned} F=\prod _{E^{*}\in {\mathcal {E}}}\prod _{\begin{array}{c} E \text {~is a regular irreducible}\\ \text {~erasure pattern of the same type with~} E^{*} \end{array}}f_E, \end{aligned}$$

(19)

and for each $E^{*}=(E_1,E_2,\ldots ,E_m)^{T}\in {\mathcal {E}}$, we can find at most ${n\atopwithdelims ()|V_E|}\le {n\atopwithdelims ()bm-b}$ different regular irreducible erasure patterns of type $E^{*}$ in $[m]\times [n]$. By (17), for every regular irreducible erasure pattern E, we have the degree of each variable $x_{ij}$ in $f_{E}$ to be at most $m-1$. Therefore, the degree of each variable $x_{ij}$ in F is at most $(m-1)\cdot |{\mathcal {E}}|\cdot {n\atopwithdelims ()bm-b}$.

Now, consider the polynomial $P\cdot F$, by Lemma 4.2, there is a valuation $(h_{11},\ldots ,h_{bn})$ of $(x_{11},\ldots ,x_{bn})$ over a field ${\mathbb {F}}_q$ of size

$$\begin{aligned} q&=(m-1)\cdot {m\cdot b(m-1)\atopwithdelims ()2b(m-1)}\cdot n^{bm-b}+n^{b-1} \end{aligned}$$

such that $P\cdot F(h_{11},\ldots ,h_{bn})\ne 0$. Therefore, the corresponding matrix ${\mathbf {H}}_{row}=(h_{ij})_{i\in [b],j\in [n]}$ is the objective matrix satisfying both requirement (i) and requirement (ii). This completes the proof.

Remark 4.3

In the proof above, according to the property of regular irreducible erasure patterns, we can bound the degree of each variable in F. During this bounding process, we also showed that $\deg {(f_E)}\le b(m-1)$, which will lead to an upper bound on $\deg {(F)}$. Based on this, one can apply Schwartz-Zippel lemma with $P\cdot F$ and obtain $q\le (bm-b)\cdot {m\cdot b(m-1)\atopwithdelims ()2b(m-1)}\cdot n^{bm-b}+n^{b-1}$, which is slightly weaker than the bound given in Theorem 4.1.

Remark 4.4

Considering the MR codes for topologies $T_{m\times n}(1,b,0)$, the general bound given by Gopalan et al. [13] is

$$\begin{aligned} q>(n+bm-b)\cdot {mn\atopwithdelims (){\le n+bm-b}}=\Omega ((n+bm-b)^2(\frac{mn}{n+bm-b})^{(n+bm-b)}),\nonumber \\ \end{aligned}$$

(20)

which is exponentially increasing for both m and n, while the bound given by Theorem 4.1 is only a polynomial of n.

But, even so, when considering the growth rate corresponding to m,

$$\begin{aligned} q=(m-1)\cdot {m\cdot b(m-1)\atopwithdelims ()2b(m-1)}\cdot n^{bm-b}+n^{b-1}=\Omega (m^{2b(m-1)+m}n^{bm-b}) \end{aligned}$$

grows exponentially.

Actually, m is often considered as the number of data centers in practice, which is very small compared to n. Therefore, the when $n\gg m$, the bound given by Theorem 4.1 is better than that in [13].

5 MR codes for topologies $T_{m\times n}(1,2,0)$ and $T_{m\times n}(1,3,0)$

In this section, for topology $T_{m\times n}(1,2,0)$, we give a full characterization of the regular irreducible erasure patterns when $m=4$ and obtain an improved upper bound on the field size for MR codes instantiating topology $T_{4\times n}(1,2,0)$. Based on a special type of regular irreducible erasure patterns, we prove a polynomial lower bound for general m. For topology $T_{m\times n}(1,3,0)$, with the same method, we obtain an improved upper bound on the field size for MR codes instantiating topology $T_{3\times n}(1,3,0)$.

5.1 MR codes for topologies $T_{m\times n}(1,2,0)$

In this part, first, using the results from Section 3.2, we will give a characterization of the regular irreducible erasure patterns for topology $T_{m\times n}(1,2,0)$. For $m=4$, all types of regular irreducible erasure patterns are determined. For each $m\ge 5$, we find a particular type of regular irreducible erasure patterns, which is crucial for the proof of the lower bound.

Denote ${\mathcal {E}}$ as the set of all the types of regular irreducible erasure patterns for topology $T_{m\times n}(1,2,0)$. For each $E\in {\mathcal {E}}$, by (13) and (14), we have $|U_E|+2\le |V_E|\le 2|U_E|-2$ and $|E|\le |V_E|+2|U_E|-2$. Therefore, $|U_E|\ge 4$ and by $U_E\subseteq [m]$, we have $m\ge 4$.

From the irreducibility, each erasure pattern has at least 2 erasures in each column and at least 3 erasures in each row. Noted that the more erasures each row (column) contains, the harder the corresponding erasure pattern can meet the regularity. In this spirit, we find the following 2 types of erasure patterns for $m=4$.

• Type 1

  $$\begin{aligned} E_1=\left( \begin{array}{cccccc} * &{} * &{} * &{} \circ &{} \circ &{} \circ \\ * &{} * &{} \circ &{} * &{} \circ &{} \circ \\ \circ &{} \circ &{} * &{} \circ &{} * &{} * \\ \circ &{} \circ &{} \circ &{} * &{} * &{} * \\ \end{array} \right) _{4\times 6}, \end{aligned}$$

• Type 2

  $$\begin{aligned} E_2=\left( \begin{array}{cccccc} * &{} * &{} * &{} \circ &{} \circ &{} \circ \\ * &{} \circ &{} \circ &{} * &{} * &{} \circ \\ \circ &{} * &{} \circ &{} * &{} \circ &{} * \\ \circ &{} \circ &{} * &{} \circ &{} * &{} * \\ \end{array} \right) _{4\times 6}. \end{aligned}$$

Proposition 5.1

The above 2 types of erasure patterns are regular and irreducible for corresponding topology $T_{m\times n}(1,2,0)$.

Proof

Noted that the irreducibility follows from that all these erasure patterns have at least 2 erasures in each column and at least 3 erasures in each row, thus we only have to prove the regularity. We just prove the regularity of erasure patterns of Type 1 here, the proof for the other type is the same.

Recall that erasure pattern E is called regular if for all $U\subseteq [m]$ and $V\subseteq [n]$, we have

$$\begin{aligned} |E\cap (U\times V)|\le |V|+2|U|-2. \end{aligned}$$

Since each submatrix indexed by $U\times V\subseteq [m]\times [n]$ can be obtained by executing first $m-|U|$ row deletions and then $n-|V|$ column deletions, and the violation of the above inequality only can occur when E has the maximum density in $[m]\times [n]$. Therefore, w.l.o.g., for erasure patterns of Type 1, assume that $m=4$ and $n=6$.

Consider the row and column deletion process in $E_1$. Every deletion of $4-|U|$ rows leads to a decrease of $12-3|U|$ in the LHS and $8-2|U|$ in the RHS, thus to maintain the inequality, the following column deletions can contribute at most $4-|U|$ more decreases in the LHS. Noted that each pair of rows shares at most 2 erasures having common coordinates. Therefore, for $|U|\le 3$, the column deletions can contribute at most $4-|U|$ more decreases in the LHS than that in the RHS. Therefore, the above inequality always holds and this leads to the regularity of erasure patterns of Type 1. $\square$

Remark 5.2

When $m>4$, since $4\le |U_E|\le m$ and $|U_{E}|+2\le |V_{E}|\le 2|U_{E}|-2$, the type of erasure patterns varies with the growing of $|U_{E}|$. In the same spirit, one can construct similar regular irreducible erasure patterns for $m=5,6,7$. And with the help of Proposition 3.4, one can extend these constructions recursively for general m.

5.1.1 Improved upper bound for topologies $T_{4\times n}(1,2,0)$

Now, we are going to prove the following existence result for MR codes instantiating the topology $T_{4\times n}(1,2,0)$, which improves the general upper bound from Theorem 4.1 for this special topology.

Theorem 5.3

For any $q>\frac{n^{5}}{\log (n)}\cdot C_1$, there exists an MR code ${\mathcal {C}}$ that instantiates the topology $T_{4\times n}(1,2,0)$ over the field ${\mathbb {F}}_q$, where $C_1\ge (\frac{10}{c_5})^5$ is an absolute constant.

Proof

Similar to the proof of Theorem 4.1, let ${\mathcal {C}}_{col}$ be the simple parity code ${\mathcal {P}}_4$. Our goal is to construct a $2\times n$ matrix ${\mathbf {H}}_{row}$ such that:

• (i) Every 2 distinct columns of ${\mathbf {H}}_{row}$ are linearly independent.

• (ii) For each regular irreducible erasure pattern E of Type I or Type II, the pseudo-parity check matrix ${\mathbf {H}}\in {\mathbb {F}}_q^{(n+8)\times 4n}$ of ${\mathcal {P}}_4\otimes {\mathcal {C}}_{row}$ satisfies: $rank({\mathbf {H}}|_{E})=12$.

Different from the general strategy, we are going to obtain an objective matrix based on the Vandermonde matrix.

Suppose there exists an objective matrix ${\mathbf {A}}_{0}$ of the form

$$\begin{aligned} {\mathbf {A}}_{0}=\left( \begin{array}{ccccc} 1 &{} 1 &{} 1 &{} \cdots &{} 1 \\ a_1 &{} a_2 &{} a_3 &{} \cdots &{} a_n \\ \end{array} \right) , \end{aligned}$$

where $\{a_i\}_{i\in [n]}$ are pairwise distinct elements in ${\mathbb {F}}_q$. Then the distinctness of $\{a_i\}_{i\in [n]}$ guarantees that ${\mathbf {A}}_{0}$ satisfies (i).

Now take ${\mathbf {H}}_{row}={\mathbf {A}}_0$ and consider the pseudo-parity check matrix $\mathbf {H_{A_0}}$. For each $s\in [2]$, we have

$$\begin{aligned} \mathbf {H_{A_0}}|_{E_s}=\left( \begin{array}{cc} {\mathbf {I}}_{6} &{} {\mathbf {0}}_{6 \times 6}\\ {\mathbf {A}}^{(s)}_{8 \times 6} &{} {\mathbf {B}}^{(s)}_{8\times 6}\\ {\mathbf {0}}_{(n-6)\times 6} &{} {\mathbf {0}}_{(n-6)\times 6}\\ \end{array} \right) , \end{aligned}$$

where

and

$$\begin{aligned} {\mathbf {B}}^{(2)}= \left( \begin{array}{cccccc} -1 &{} -1 &{} -1 &{} &{} &{} \\ -a_1 &{} -a_2 &{} -a_3 &{} &{} &{} \\ 1 &{} &{} &{} -1 &{} -1 &{} \\ a_1&{} &{} &{} -a_4 &{} -a_5 &{} \\ &{} 1 &{} &{} 1 &{} &{} -1\\ &{} a_2 &{} &{} a_4 &{} &{} -a_6\\ &{} &{} 1 &{} &{} 1 &{} 1\\ &{} &{} a_3 &{} &{} a_5 &{} a_6\\ \end{array} \right) . \end{aligned}$$

Since ${\mathbf {B}}^{(s)}$ can be simplified as

and

we have

• $rank({\mathbf {B}}^{(1)})=6$ if and only if $(a_2-a_1)(a_4-a_3)(a_6-a_5)\ne 0$.

• $rank({\mathbf {B}}^{(2)})=6$ if and only if $(a_1-a_4)(a_2-a_6)(a_3-a_5)-(a_2-a_4)(a_1-a_5)(a_3-a_6)\ne 0$.

Take $f(x_1,x_2,\ldots ,x_6)=(x_1-x_4)(x_2-x_6)(x_3-x_5)-(x_2-x_4)(x_1-x_5)(x_3-x_6)$, we have $deg(f)=3$. From the assumption that $\{a_i\}_{i\in [n]}$ are pairwise distinct, we know that erasure patterns of Type I can be easily corrected. Then if we also want to correct all erasure patterns of Type II, $\{a_i\}_{i\in [n]}$ only need to have the property that for any $\{a_{i_1},a_{i_2},\ldots ,a_{i_6}\}\subseteq \{a_i\}_{i\in [n]}$ and each $\pi \in S_6$, $f(a_{i_{\pi (1)}},\ldots ,a_{i_{\pi (6)}})\ne 0$.

Different from the proof of Theorem 4.1, here we use the hypergraph independent set approach.

Let ${\mathcal {H}}$ be a 6-uniform hypergraph with vertex set ${\mathbb {F}}_q$, each set of 6 vertices $\{v_{1},\ldots ,v_{6}\}$ forms a 6-hyperedge if and only if $f(v_{\pi (1)},\ldots ,v_{\pi (6)})=0$ for some $\pi \in S_{6}$. From the construction of the hypergraph ${\mathcal {H}}$, if there exists an independent set I such that $|I|\ge n$, then we can construct an objective matrix ${\mathbf {A}}_{0}$ by arbitrarily choosing n different vertices from I as elements for its $2_{nd}$ row.

Since $deg_{x_i}(f)=1$ for each $x_i$, and $f(v_1,\ldots ,x_i,\ldots ,v_6)$ is a non-zero polynomial for any 5-subset $\{v_{j}\}_{j\in [6]\setminus \{i\}}\subseteq {\mathbb {F}}_q$. Thus the maximal 5-degree of ${\mathcal {H}}$ $\Delta _{5}({\mathcal {H}})\le 6!$. By Theorem 2.10,

$$\begin{aligned} \alpha ({\mathcal {H}})\ge \frac{c_5}{5}(q\log {q})^{\frac{1}{5}}>n. \end{aligned}$$

Denote $I({\mathcal {H}})$ as the maximum independent set in ${\mathcal {H}}$, therefore, there exists a subset $A=\{a_1\ldots ,a_n\}\subseteq {\mathbb {F}}_q$ such that the matrix ${\mathbf {A}}_0$ of the form

$$\begin{aligned} {\mathbf {A}}_{0}=\left( \begin{array}{ccccc} 1 &{} 1 &{} 1 &{} \cdots &{} 1 \\ a_1 &{} a_2 &{} a_3 &{} \cdots &{} a_n \\ \end{array} \right) \end{aligned}$$

satisfies both (i) and (ii). Thus, the resulting tensor product code ${\mathcal {C}}={\mathcal {P}}_{4}\otimes {\mathcal {C}}_{row}$ is an MR code instantiating topology $T_{4\times n}(1,2,0)$. $\square$

5.1.2 Lower bound for topologies $T_{m\times n}(1,2,0)$

The above theorem says that for any $q>\frac{n^{5}}{\log (n)}\cdot C_0$, there exists an MR code ${\mathcal {C}}$ for topology $T_{4\times n}(1,2,0)$ over ${\mathbb {F}}_q$. This actually gives an upper bound $\frac{n^{5}}{\log (n)}\cdot C_0$ on the minimal field size required for the existence of an MR code. But is this polynomial trend really necessary? Recall the MDS Conjecture:

MDS Conjecture

If there is a nontrivial [n, k] MDS code over ${\mathbb {F}}_q$, then $n\le q+1$, except when q is even and $k=3$ or $k=q-1$ in which case $n\le q+2$.

Since the code ${\mathcal {C}}_{row}$ is always an MDS code, thus from the MDS Conjecture we know that a linear lower bound is necessary, but will it be sufficient? Sadly not. The next theorem gives a quadratic lower bound on the smallest field size required for the existence of an MR code for the topology $T_{m\times n}(1,2,0)$ ($m\ge 4$ and $n\ge 6$).

Theorem 5.4

If $q<\frac{(n-3)^{2}}{4}+2$, then for any tensor product code ${\mathcal {C}}={\mathcal {C}}_{col}\otimes {\mathcal {C}}_{row}$ over ${\mathbb {F}}_q$ with ${\mathcal {C}}_{col}$ as an $[m,m-1,2]$ MDS code ($m\ge 4$) and ${\mathcal {C}}_{row}$ as an $[n,n-2,3]$ MDS code ($n\ge 6$), ${\mathcal {C}}$ can not be an MR code that instantiates the topology $T_{m\times n}(1,2,0)$.

To present the proof, we need the following two propositions:

Proposition 5.5

Take $\omega \in {\mathbb {F}}_q^*$ as the primitive element. If there exist six distinct $t_i\in {\mathbb {Z}}_{q-1}$ such that $t_1+t_6=t_2+t_5=t_3+t_4$, then the polynomial $f(x_1,x_2,\ldots ,x_6)=(x_1-x_4)(x_2-x_6)(x_3-x_5)-(x_2-x_4)(x_1-x_5)(x_3-x_6)$ has a zero of the form $(\omega ^{t_1},\ldots ,\omega ^{t_6})$.

Proof

By substituting $(\omega ^{t_1},\ldots ,\omega ^{t_6})$ to $f(x_1,x_2,\ldots ,x_6)$ directly, we have

$$\begin{aligned}&(\omega ^{t_1}-\omega ^{t_5})(\omega ^{t_2}-\omega ^{t_4})(\omega ^{t_3}-\omega ^{t_6})-(\omega ^{t_1}-\omega ^{t_4})(\omega ^{t_2}-\omega ^{t_6})(\omega ^{t_3}-\omega ^{t_5})\\&\quad = \omega ^{t_1+t_2+t_3}[(1-\omega ^{t_5-t_1})(1-\omega ^{t_4-t_2})(1-\omega ^{t_6-t_3})\\&\qquad -(1-\omega ^{t_6-t_2})(1-\omega ^{t_5-t_3})(1-\omega ^{t_4-t_1})]. \end{aligned}$$

Since $t_1+t_6=t_2+t_5=t_3+t_4$, then we have

$$\begin{aligned} {\left\{ \begin{array}{ll} t_5-t_1=t_6-t_2\\ t_4-t_2=t_5-t_3\\ t_6-t_3=t_4-t_1 \end{array}\right. }. \end{aligned}$$

Using these three identities, we have $f(\omega ^{t_1},\ldots ,\omega ^{t_6})=0$. $\square$

Let $N\ge 2$ be a positive integer, for any subset $A\subseteq {\mathbb {Z}}_N$, we say A is a 2-Sidon set if for any 2-subset $\{a_1,b_1\}\subseteq A$ there exists at most one other $\{a_2,b_2\}\subseteq A$ different from $\{a_1,b_1\}$ such that $a_1+b_1=a_2+b_2$.

Proposition 5.6

For any $A\subseteq {\mathbb {Z}}_N$, if A is a 2-Sidon set, then we have $|A|\le 2\sqrt{N}+1$.

Proof

Since $A+A\subseteq {\mathbb {Z}}_N$, by a simple double counting, we have

$$\begin{aligned} {|A|\atopwithdelims ()2}\le 2N. \end{aligned}$$

Thus $|A|\le 2\sqrt{N}+1$. $\square$

Proof of Theorem 5.4

Different from the proof of the upper bound, since we want to obtain a necessary condition for the existence of an MR code, we have to deal with the general case.

For any $[m,m-1,2]$ MDS code ${\mathcal {C}}_1$ and $[n,n-2,3]$ MDS code ${\mathcal {C}}_2$, take

$$\begin{aligned} {\mathbf {H}}_1=(a_1,a_2,\ldots ,a_m)~\text {and}~ {\mathbf {H}}_2=\left( \begin{array}{cccc} b_{11} &{} b_{12} &{} \cdots &{} b_{1n} \\ b_{21} &{} b_{22} &{} \cdots &{} b_{2n} \\ \end{array} \right) \end{aligned}$$

as their parity check matrices. Then the pseudo-parity check matrix of ${\mathcal {C}}={\mathcal {C}}_1\otimes {\mathcal {C}}_2$ has the following form

$$\begin{aligned} {\mathbf {H}}=\left( \begin{array}{cccc} a_1\cdot {\mathbf {I}}_n &{} a_2\cdot {\mathbf {I}}_n &{} \cdots &{} a_m\cdot {\mathbf {I}}_n \\ {\mathbf {H}}_{2} &{} {\mathbf {0}} &{} \cdots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {H}}_{2} &{} \cdots &{} {\mathbf {0}} \\ \vdots &{} \vdots &{} \ddots &{} {\mathbf {0}} \\ {\mathbf {0}} &{} {\mathbf {0}} &{} \cdots &{} {\mathbf {H}}_{2} \\ \end{array} \right) . \end{aligned}$$

Take ${\mathbf {H}}_1=(a_1,a_2,\ldots ,a_m)$ as a vector in ${\mathbb {F}}_q$ and consider its Hamming weight $w({\mathbf {H}}_1)$. Since ${\mathcal {C}}_1$ is an $[m,m-1,2]$ MDS code, we have $w({\mathbf {H}}_1)=m$. Therefore, $a_i\ne 0$ for each $i\in [m]$.

Now, consider erasure patterns of type 2, we have

$$\begin{aligned} {\mathbf {H}}|_{E_2}=\left( \begin{array}{cc} \mathbf {A'}_{6 \times 6} &{} {\mathbf {0}}_{6 \times 6}\\ {\mathbf {A}}_{8 \times 6} &{} {\mathbf {B}}_{8\times 6}\\ {\mathbf {0}}_{(n-6)\times 6} &{} {\mathbf {0}}_{(n-6)\times 6}\\ \end{array} \right) , \end{aligned}$$

where

and

$$\begin{aligned} {\mathbf {B}}= \left( \begin{array}{cccccc} -\frac{a_2'}{a_1'}\beta _1 &{} -\frac{a_3'}{a_1'}\beta _2 &{} -\frac{a_4'}{a_1'}\beta _3 &{} &{} &{} \\ \beta _1 &{} &{} &{} -\frac{a_3'}{a_2'}\beta _4 &{} -\frac{a_4'}{a_2'}\beta _5 &{} \\ &{} \beta _2 &{} &{} \beta _4 &{} &{} -\frac{a_4'}{a_3'}\beta _6\\ &{} &{} \beta _3 &{} &{} \beta _5 &{} \beta _6\\ \end{array} \right) , \end{aligned}$$

for some $\{a_1',a_2',a_3',a_4'\}\subseteq \{a_1,a_2,\ldots ,a_m\}$ and column vectors $\{\beta _1,\ldots ,\beta _6\}\subseteq {\mathbf {H}}_2$ corresponding to $E_2$.

It can be easily verified that the first row of ${\mathbf {B}}$ can be linearly expressed by the other three rows. Thus $rank({\mathbf {B}})=6$ if and only if the following system of linear equations only have zero solutions.

$$\begin{aligned} {\left\{ \begin{array}{ll} x_1\cdot \beta _1-x_4\cdot \frac{a_3'}{a_2'}\beta _4-x_5\cdot \frac{a_4'}{a_2'}\beta _5=0 \\ x_2\cdot \beta _2+x_4\cdot \beta _4-x_6\cdot \frac{a_4'}{a_3'}\beta _6=0 \\ x_3\cdot \beta _3+x_5\cdot \beta _5+x_6\cdot \beta _6=0 \\ \end{array}\right. } \end{aligned}$$

(21)

For (21), it has non-zero solution $(d_1,\ldots ,d_6)$ which does not violate the MDS property of ${\mathbf {H}}_2$. For example, take $\omega \in {\mathbb {F}}_q^*$ as the primitive element, if $\beta _i=(1, \omega ^{t_i})$ for some distinct $t_i\in [q-1]$ such that $t_1+t_6=t_2+t_5=t_3+t_4$, then the resulting ${\mathbf {B}}_2$ has $rank({\mathbf {B}}_2)\le 5$ and this guarantees the existence of non-zero solution for (21).

W.o.l.g., assume $n\ge 8$, then from the MDS property of ${\mathbf {H}}_2$, we know that ${\mathbf {H}}_2$ contains at least $n-2$ weight-2 columns. Since any six distinct elements of [n] can be chosen to form an erasure pattern $E_2$ of Type II, therefore, the maximal recoverability requires that $rank({\mathbf {B}})=6$ for any six distinct columns in ${\mathbf {H}}_2$. Especially, we can take all these columns with weight equal to 2. Assume $\beta _i=(b_{i1},b_{i2})^{T}$ with $b_{is}\ne 0$ for each $i\in [6],s\in [2]$. Then ${\mathbf {B}}$ can be formulated as

$$\begin{aligned} {\mathbf {B}}= \left( \begin{array}{@{}cccccc@{}} -\frac{a_2' b_{11}}{a_1}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_1\end{array}\right) &{} -\frac{a_3' b_{21}}{a_1}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_2\end{array}\right) &{} -\frac{a_4' b_{31}}{a_1}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_3\end{array}\right) &{} &{} &{} \\ b_{11}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_1\end{array}\right) &{} &{} &{} -\frac{a_3' b_{41}}{a_2}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_4\end{array}\right) &{} -\frac{a_4' b_{51}}{a_2}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_5\end{array}\right) &{} \\ &{} b_{21}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_2\end{array}\right) &{} &{} b_{41}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_4\end{array}\right) &{} &{} -\frac{a_4' b_{61}}{a_3}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_6\end{array}\right) \\ &{} &{} b_{31}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_3\end{array}\right) &{} &{} b_{51}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_5\end{array}\right) &{} b_{61}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_6\end{array}\right) \\ \end{array} \right) , \end{aligned}$$

where $r_i=\frac{b_{i2}}{b_{i1}}$ for each $i\in [6]$. Since the first row of ${\mathbf {B}}$ can be linearly expressed by the other three rows and the scaling of each column doesn’t affect the linear dependency, we have $rank({\mathbf {B}})=rank(\tilde{{\mathbf {B}}})$, where

$$\begin{aligned} \tilde{{\mathbf {B}}}= \left( \begin{array}{@{}cccccc@{}} \left( \begin{array}{@{}c@{}} 1 \\ r_1\end{array}\right) &{} &{} &{} -\frac{a_3'}{a_2'}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_4\end{array}\right) &{} -\frac{a_4'}{a_2'}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_5\end{array}\right) &{} \\ &{}\left( \begin{array}{@{}c@{}} 1 \\ r_2\end{array}\right) &{} &{} \left( \begin{array}{@{}c@{}} 1 \\ r_4\end{array}\right) &{} &{} -\frac{a_4'}{a_3'}\cdot \left( \begin{array}{@{}c@{}} 1 \\ r_6\end{array}\right) \\ &{} &{} \left( \begin{array}{@{}c@{}} 1 \\ r_3\end{array}\right) &{} &{} \left( \begin{array}{@{}c@{}} 1 \\ r_5\end{array}\right) &{} \left( \begin{array}{@{}c@{}} 1 \\ r_6\end{array}\right) \\ \end{array} \right) . \end{aligned}$$

And it can be simplified as

$$\begin{aligned} \tilde{{\mathbf {B}}}=\left( \begin{array}{cccccc} 1 &{} &{} &{} &{} &{} \\ &{} 1 &{} &{} &{} &{} \\ &{} &{} 1 &{} &{} &{} \\ &{} &{} &{} \frac{a_3'}{a_2'}(r_1-r_4) &{} \frac{a_4'}{a_2'}(r_1-r_5) &{} \\ &{} &{} &{} r_4-r_2 &{} &{} \frac{a_4'}{a_3'}(r_2-r_6)\\ &{} &{} &{} &{} r_5-r_3 &{} r_6-r_3\\ \end{array} \right) . \end{aligned}$$

Therefore, $rank({\mathbf {B}})=rank(\tilde{{\mathbf {B}}})=6$ if and only if $(r_1-r_4)(r_2-r_6)(r_3-r_5)-(r_2-r_4)(r_1-r_5)(r_3-r_6)\ne 0$.¹

In order to show that the tensor product code ${\mathcal {C}}$ can’t correct all erasure patterns of Type II, we need to prove that if q isn’t large enough, there will always be six distinct columns $\{b_{i1}\cdot (1,r_i)^{T}\}_{i\in [6]}$ with $(r_1-r_4)(r_2-r_6)(r_3-r_5)-(r_2-r_4)(r_1-r_5)(r_3-r_6)=0$, which is shown as follows.

Consider $n-2$ distinct columns of ${\mathbf {H}}_2$ with weight 2, $\{b_{i1}\cdot (1,r_i)^{T}\}_{i\in [n-2]}$, according to the MDS property, we know that $r_i\ne r_j$ for all $i\ne j\in [n-2]$. Therefore, if we take $r_i=\omega ^{t_i}$ for each $i\in [n-2]$, we know that $t_i\ne t_j$ for all $i\ne j\in [n-2]$. Denote $A=\{t_i\}_{i\in [n-2]}$, then A is an $(n-2)$-subset of ${\mathbb {Z}}_{q-2}$. Since $q< \frac{(n-3)^2}{4}+2$, by Proposition 5.6, we know that A can’t be a 2-Sidon set. Thus, there are at least three different 2-subsets $\{t_1,t_6\},\{t_2,t_5\},\{t_3,t_4\}\in A$, such that $t_1+t_6=t_2+t_5=t_3+t_4$ and $t_i$s are all distinct. By Proposition 5.5, the corresponding $\{r_j\}_{j\in [6]}$ such that $r_j=\omega ^{t_j}$ for each $j\in [6]$, satisfies $(r_1-r_4)(r_2-r_6)(r_3-r_5)-(r_2-r_4)(r_1-r_5)(r_3-r_6)=0$.

Therefore, ${\mathcal {C}}$ can not correct the erasure patterns of Type II formed by the corresponding six columns $\{b_{j1}\cdot (1,r_j)^{T}\}_{j\in [6]}$, which means ${\mathcal {C}}$ is not an MR code that instantiates the topology $T_{m\times n}(1,2,0)$. $\square$

Remark 5.7

Since we have only considered the restrictions brought by certain erasure patterns, the lower bounds of the field size given by Theorem 5.4 can certainly be improved if one can find other non-trivial regular irreducible erasure patterns for general m.

5.2 MR codes for topologies $T_{m\times n}(1,3,0)$

Denote ${\mathcal {E}}$ as the set of all the types of regular irreducible erasure patterns for topology $T_{m\times n}(1,3,0)$. For each $E\in {\mathcal {E}}$, by (13), we have $|U_E|+3\le |V_E|\le 3|U_E|-3$, which leads to $|U_E|\ge 3$. Since $U_E\subseteq [m]$, we have $m\ge 3$ and $n\ge 6$.

From the irreducibility, each erasure pattern has exactly 2 erasures in each column and 4 erasures in each row. With the same spirit as that for topologies $T_{m\times n}(1,2,0)$, we find the following type of erasure patterns for $m=3$:

$$\begin{aligned} E_0=\left( \begin{array}{cccccc} * &{} * &{} * &{} * &{} \circ &{} \circ \\ * &{} * &{} \circ &{} \circ &{} * &{} * \\ \circ &{} \circ &{} * &{} * &{} * &{} * \\ \end{array} \right) . \end{aligned}$$

Using the same argument as that for $E_1$ and $E_2$, one can prove that erasure patterns of type $E_0$ are regular and irreducible for topologies $T_{m\times n}(1,3,0)$. Also, since $m=3$ leads to $U_E=3$, $V_E=6$ and $|E|=12$ for every $E\in {\mathcal {E}}$, one can easily check that $E_0$ is the only type of regular irreducible erasure pattern for topologies $T_{3\times n}(1,3,0)$.

Based on this characterization, we have the following improved upper bound on the field size required for the existence of MR codes instantiating $T_{3\times n}(1,3,0)$.

Theorem 5.8

For any $q>\frac{n^{5}}{\log (n)}\cdot C_2$, there exists an MR code ${\mathcal {C}}$ that instantiates the topology $T_{3\times n}(1,3,0)$ over the field ${\mathbb {F}}_q$, where $C_2\ge (\frac{10}{c_5})^5$ is an absolute constant.

Proof

(Sketch of the proof) Since the idea of the proof is the same as that of Theorem 5.3, we only sketch the main steps here.

Let ${\mathcal {C}}_{col}$ be the simple parity code ${\mathcal {P}}_3$, we are going to construct a $3\times n$ matrix ${\mathbf {H}}_{row}$ such that:

• (i) Every 3 distinct columns of ${\mathbf {H}}_{row}$ are linearly independent.

• (ii) For each erasure pattern E of type $E_0$, the pseudo-parity check matrix ${\mathbf {H}}\in {\mathbb {F}}_q^{(n+9)\times 3n}$ of ${\mathcal {P}}_3\otimes {\mathcal {C}}_{row}$ satisfies: $rank({\mathbf {H}}|_{E})=12$.

Suppose there exists an objective matrix ${\mathbf {A}}_{0}$ of the form

$$\begin{aligned} {\mathbf {A}}_{0}=\left( \begin{array}{ccccc} 1 &{} 1 &{} 1 &{} \cdots &{} 1 \\ a_1 &{} a_2 &{} a_3 &{} \cdots &{} a_n \\ a_1^2 &{} a_2^2 &{} a_3^2 &{} \cdots &{} a_n^2 \\ \end{array} \right) , \end{aligned}$$

where $\{a_i\}_{i\in [n]}$ are pairwise distinct elements in ${\mathbb {F}}_q$. Then we have the corresponding pseudo-parity check matrix

$$\begin{aligned} \mathbf {H_{A_0}}|_{E_0}=\left( \begin{array}{cc} {\mathbf {I}}_{6} &{} {\mathbf {0}}_{6 \times 6}\\ {\mathbf {A}}_{9 \times 6} &{} {\mathbf {B}}_{9\times 6}\\ {\mathbf {0}}_{(n-6)\times 6} &{} {\mathbf {0}}_{(n-6)\times 6}\\ \end{array} \right) , \end{aligned}$$

where

Since ${\mathbf {B}}$ can be simplified as

thus we have

• $rank({\mathbf {B}})=6$ if and only if $f(a_1,\ldots ,a_6)\ne 0$, where $f(x_1,\ldots ,x_6)=(x_1-x_2)(x_3-x_4)[(x_1-x_6)(x_2-x_6)(x_3-x_5)(x_4-x_6)-(x_1-x_5)(x_2-x_5)(x_3-x_6)(x_4-x_6)]$.

Let ${\mathcal {H}}$ be a 6-uniform hypergraph with vertex set ${\mathbb {F}}_q$, each set of 6 vertices $\{v_{1},\ldots ,v_{6}\}$ forms a 6-hyperedge if and only if $f(v_{\pi (1)},\ldots ,v_{\pi (6)})=0$ for some $\pi \in S_{6}$. If there exists an independent set $I\subseteq {\mathbb {F}}_q$ such that $|I|\ge n$, then we can construct an objective matrix ${\mathbf {A}}_{0}$ by arbitrarily choosing n different vertices from I as $\{a_i\}_{i\in [n]}$.

Since $deg_{x_i}(f)\le 2$ for each $x_i$, and $f(v_1,\ldots ,x_i,\ldots ,v_6)$ is a non-zero polynomial for any 5-subset $\{v_{j}\}_{j\in [6]\setminus \{i\}}\in {\mathbb {F}}_q$, we have $\Delta _{5}({\mathcal {H}})\le 2\cdot 6!$. Thus, by Theorem 2.10,

$$\begin{aligned} \alpha ({\mathcal {H}})\ge \frac{c_5}{5}(q\log {q})^{\frac{1}{5}}>n, \end{aligned}$$

therefore, there exists a subset $A=\{a_1,\ldots ,a_n\}\subseteq {\mathbb {F}}_q$ such that the corresponding Vandermonde matrix ${\mathbf {A}}_0$ is the objective parity check matrix of the row code ${\mathcal {C}}_{row}$. $\square$

Remark 5.9

Based on the same idea as that appeared in the proof of Theorem 5.4, we can obtain a lower bound $q\ge \frac{\sqrt{n^2-11n+34}}{2}$ on the field size ensuring the existence of MR codes for topologies $T_{m\times n}(1,3,0)$. Unfortunately, this doesn’t beat the lower bound $q\ge n-1$ or $q\ge n-2$ given by the MDS conjecture.

6 Concluding Remarks and Further Research

In this paper, we obtain a new upper bound on the minimal size of fields required for the existence of MR codes that instantiate the topology $T_{m\times n}(1,b,0)$, which improves the general upper bound given by Gopalan et al. [13]. We also consider some special cases with fixed m and b, and obtain a polynomial lower bound and some new upper bounds. Though many works have been done, there is still a wide range of questions that remain open. Here we highlight some of the questions related to our work.

• Due to the rough estimation on the number of regular erasure patterns, the upper bound given by Theorem 4.1 still grows exponentially with m. If one can give a better characterization of the regular erasure patterns (probably using tools from extremal graph theory), we believe the general upper bound in Theorem 4.1 can also be improved.

• As for the lower bound, we only considered the cases when $b=2$ and 3. For general case, due to the complexity of the erasure patterns, our method might not work. Therefore, a general non-trivial lower bound on the field size of codes achieving the MR property for topologies $T_{m\times n}(1,b,0)$ remains widely open.

• Under the limitations of the methods themselves, the Combinatorial Nullstellensatz and the hypergraph independent set approach can only give existence results. Therefore, explicit constructions of MR codes for topologies $T_{m\times n}(1,b,0)$ over small fields are still interesting open problems. In particular, is it possible to give an explicit construction of MR codes for the topology $T_{4\times n}(1,2,0)$ over a field of size between $\Omega (n^2)$ and ${\mathcal {O}}(n^5/\log (n))$?