Operator Scaling: Theory and Applications

Abstract

In this paper, we present a deterministic polynomial time algorithm for testing whether a symbolic matrix in non-commuting variables over ${\mathbb {Q}}$ is invertible or not. The analogous question for commuting variables is the celebrated polynomial identity testing (PIT) for symbolic determinants. In contrast to the commutative case, which has an efficient probabilistic algorithm, the best previous algorithm for the non-commutative setting required exponential time (Ivanyos et al. in Comput Complex 26(3):717–763, 2017) (whether or not randomization is allowed). The algorithm efficiently solves the “word problem” for the free skew field, and the identity testing problem for arithmetic formulae with division over non-commuting variables, two problems which had only exponential time algorithms prior to this work. The main contribution of this paper is a complexity analysis of an existing algorithm due to Gurvits (J Comput Syst Sci 69(3):448–484, 2004), who proved it was polynomial time for certain classes of inputs. We prove it always runs in polynomial time. The main component of our analysis is a simple (given the necessary known tools) lower bound on central notion of capacity of operators (introduced by Gurvits 2004). We extend the algorithm to actually approximate capacity to any accuracy in polynomial time, and use this analysis to give quantitative bounds on the continuity of capacity (the latter is used in a subsequent paper on Brascamp–Lieb inequalities). We also extend the algorithm to compute not only singularity, but actually the (non-commutative) rank of a symbolic matrix, yielding a factor 2 approximation of the commutative rank. This naturally raises a relaxation of the commutative PIT problem to achieving better deterministic approximation of the commutative rank. Symbolic matrices in non-commuting variables, and the related structural and algorithmic questions, have a remarkable number of diverse origins and motivations. They arise independently in (commutative) invariant theory and representation theory, linear algebra, optimization, linear system theory, quantum information theory, approximation of the permanent and naturally in non-commutative algebra. We provide a detailed account of some of these sources and their interconnections. In particular, we explain how some of these sources played an important role in the development of Gurvits’ algorithm and in our analysis of it here.

Symbolic Matrices with Polynomial Entries and Non-commutative Rank

In this section, we show how to compute the non-commutative rank of any (not necessarily square) matrix with linear entries over the free skew field . This will be achieved in two ways: the first, in Sect. A.2, by reducing this problem to testing singularity of a certain square matrix with linear entries, and the second, in Sect. A.3, by a purely quantum approach which in a sense mimics the reduction from maximum matching to perfect matching.

In fact, we solve a more general problem. Section A.1 starts with a reduction of computing ${\text{ nc-rank }}$ of a matrix with polynomial entries (given by formulae), to the problem of computing the ${\text{ nc-rank }}$ of a matrix with linear entries, via the so-called “Higman’s trick” (Proposition A.2). We give the simple quantitative analysis of this reduction, which as far as we know does not appear in the literature and may be useful elsewhere. This reduction, with the two above, allow computing the non-commutative rank of any matrix in time polynomial in the description of its entries.

1.1 Higman’s Trick

Before stating the full version of the effective Higman trick, we need to define the bit complexity of a formula computing a non-commutative polynomial.

Definition A.1

(Bit Complexity of a Formula) Let $\Phi$ be a non-commutative formula without divisions such that each of its gates computes a polynomial in $\mathbb {Q}\langle {\mathbf {x}}\rangle$ (i.e., the inputs to the formula are either rational numbers or non-commutative variables). The bit complexity of $\Phi$ is the maximum bit complexity of any rational input appearing in the formula $\Phi$.

With this definition in hand, we can state and prove Higman’s trick, which first appeared in [43]. In the proposition below, it will be useful to have the following notation to denote the direct sum of two matrices A and B:

$$\begin{aligned} A \oplus B = \begin{pmatrix} A &{}\quad 0 \\ 0 &{}\quad B \end{pmatrix}, \end{aligned}$$

where the zero matrices in the top right and bottom left corners are of appropriate dimensions. Before stating and proving Higman’s trick, let us work through a small example which showcases the essence of the trick.

Suppose we want to know the ${\text{ nc-rank }}$ of matrix $\begin{pmatrix} 1 &{} x\\ y &{} z + xy \end{pmatrix}$. The problem here is that this matrix is not linear, and we need to have a linear matrix. How can we convert this matrix into a linear matrix while preserving the rank, or the complement of the rank? To do this, we need to remove the multiplication happening in $z + xy$.

Notice that the complement of its rank does not change after the following transformation:

$$\begin{aligned} \begin{pmatrix} 1 &{}\quad x \\ y &{}\quad z + xy \end{pmatrix} \mapsto \begin{pmatrix} 1 &{}\quad x &{}\quad 0 \\ y &{}\quad z + xy &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 1 \end{pmatrix}. \end{aligned}$$

Since the complement of the rank does not change after we perform elementary row or column operations, we can first add $x \cdot \text {(third row)}$ to the second row, and then subtract $\text {(third column)} \cdot y$ to the second column, to obtain:

$$\begin{aligned} \begin{pmatrix} 1 &{}\quad x &{}\quad 0 \\ y &{}\quad z + xy &{}\quad 0 \\ 0 &{}\quad 0 &{}\quad 1 \end{pmatrix} \mapsto \begin{pmatrix} 1 &{}\quad x &{}\quad 0 \\ y &{}\quad z + xy &{}\quad x \\ 0 &{}\quad 0 &{}\quad 1 \end{pmatrix} \mapsto \begin{pmatrix} 1 &{}\quad x &{}\quad 0 \\ y &{}\quad z &{}\quad x \\ 0 &{}\quad -y &{}\quad 1 \end{pmatrix} \end{aligned}$$

The complement of the rank of this last matrix is the same as the complement of the rank of our original matrix! In particular, if this last matrix is full rank, it implies that our original matrix is also full rank. This is the essence of Higman’s trick. We now proceed to its full version.

Proposition A.2

(Effective Higman’s Trick) Let $A \in \mathbb {Q}\langle {\mathbf {x}}\rangle ^{m \times n}$ be a matrix where each entry $a_{ij}$ is computed by a non-commutative formula of size $\le s$ and bit complexity $\le b$ without divisions. Let k be the total number of multiplication gates used in the computation of the entries of A. There exist matrices $P \in \mathbb {GL}_{m+k}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$, $Q \in \mathbb {GL}_{n+k}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$ such that $P (A \oplus I_k) Q$ is a matrix with linear entries and coefficients with bit complexity bounded by b. Moreover, given access to the formulas computing the entries, one can construct P and Q efficiently in time ${\text{ poly }}(m, n, s, b)$. Since P and Q are non-singular matrices, the co-rank and the co-nc-rank of $P (A \oplus I_k) Q$ are the same as the co-rank and the co-nc-rank of A.

Proof

Let ${\textsf {Mult}}(a_{ij})$ be the number of multiplication gates in the formula computing entry $a_{ij}$ and

$$\begin{aligned} T = \displaystyle \sum _{\begin{array}{c} 1 \le i \le m \\ 1 \le j \le n \end{array}} {\textsf {Mult}}(a_{ij}). \end{aligned}$$

That is, T is the total number of multiplication gates used to compute all entries of the matrix A.

We prove this proposition by induction on T, for matrices of all dimensions. The base case, when $T = 0$, is trivial, as in this case A itself has linear entries. Suppose now that the proposition is true for all matrices (regardless of their dimensions) which can be computed by formulas using $< T$ multiplication gates.

Let A be our matrix, which can be computed using T multiplications. W.l.o.g., we can assume that ${\textsf {Mult}}(a_{mn}) \ge 1$. Then, by finding a multiplication gate in the formula for $a_{mn}$ that has no other multiplication gate as an ancestor, we can write $a_{mn}$ in the form $a_{mn} = a + b \cdot c$, where

$$\begin{aligned} {\textsf {Mult}}(a_{mn}) = {\textsf {Mult}}(a) + {\textsf {Mult}}(b) + {\textsf {Mult}}(c) + 1. \end{aligned}$$

Hence, the matrix

$$\begin{aligned} B = \left( I_{m-1} \oplus \begin{pmatrix} 1 &{}\quad b \\ 0 &{}\quad 1 \end{pmatrix} \right) (A \oplus 1) \left( I_{n-1} \oplus \begin{pmatrix} 1 &{}\quad 0 \\ -c &{}\quad 1 \end{pmatrix} \right) \end{aligned}$$

is such that

$$\begin{aligned} b_{ij} = {\left\{ \begin{array}{ll} a_{ij}, \text { if } i \le m, j \le n \text { and } (i,j) \ne (m,n) \\ a, \text { if } (i,j) = (m,n) \\ b, \text { if } (i,j) = (m, n+1) \\ -c, \text { if } (i,j) = (m+1, n) \\ 1, \text { if } (i,j) = (m+1, n+1) \\ 0, \text { otherwise} \end{array}\right. } \end{aligned}$$

Therefore, the number of multiplications needed to compute B is given by

$$\begin{aligned} \displaystyle \sum _{\begin{array}{c} 1 \le i \le m+1 \\ 1 \le j \le n+1 \end{array}} {\textsf {Mult}}(b_{ij})&= \left( \displaystyle \sum _{\begin{array}{c} 1 \le i \le m \\ 1 \le j \le n \end{array}} {\textsf {Mult}}(a_{ij}) \right) - {\textsf {Mult}}(a_{mn}) \\&\quad + {\textsf {Mult}}(a) + {\textsf {Mult}}(b) + {\textsf {Mult}}(c) \\&= T - {\textsf {Mult}}(a_{mn}) + {\textsf {Mult}}(a) + {\textsf {Mult}}(b) + {\textsf {Mult}}(c) \\&= T -1 \end{aligned}$$

Since B is an $(m+1) \times (n+1)$ matrix which can be computed by using a total of $T-1$ multiplication gates, by the induction hypothesis, there exist $P' \in \mathbb {GL}_{m+1 + (T-1)}( \mathbb {Q}\langle {\mathbf {x}}\rangle ) = \mathbb {GL}_{m+T}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$ and $Q' \in \mathbb {GL}_{n+1+(T-1)}( \mathbb {Q}\langle {\mathbf {x}}\rangle ) = \mathbb {GL}_{n+T}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$ such that $P'(B \oplus I_{T-1})Q'$ is a linear matrix. Since

$$\begin{aligned} B \oplus I_{T-1}&= \left( I_{m-1} \oplus \begin{pmatrix} 1 &{}\quad b \\ 0 &{}\quad 1 \end{pmatrix} \oplus I_{T-1} \right) (A \oplus I_T) \left( I_{n-1} \oplus \begin{pmatrix} 1 &{}\quad 0 \\ -c &{}\quad 1 \end{pmatrix} \oplus I_{T-1} \right) \\&= R(A \oplus I_T)S, \end{aligned}$$

where $R = \left( I_{m-1} \oplus \begin{pmatrix} 1 &{} b \\ 0 &{} 1 \end{pmatrix} \oplus I_{T-1} \right) \in \mathbb {GL}_{m+T}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$ and $S = \left( I_{n-1} \oplus \begin{pmatrix} 1 &{} 0 \\ -c &{} 1 \end{pmatrix} \oplus I_{T-1} \right) \in \mathbb {GL}_{n+T}( \mathbb {Q}\langle {\mathbf {x}}\rangle )$, we have that

$$\begin{aligned} P'(B \oplus I_{T-1})Q' = (P'R) (A \oplus I_T) (SQ'). \end{aligned}$$

Setting $P = P'R$ and $Q = SQ'$ proves the inductive step and completes the proof. Since we only use subformulas of the formulas computing the entries of A, the bound on the bit complexity does not change. $\square$

1.2 Classical Reduction

Having shown the effective version of Higman’s trick, we can now compute the ${\text{ nc-rank }}$ of a matrix over $\mathbb {Q}\langle {\mathbf {x}}\rangle$. We begin with a lemma which will tell us that we can reduce the problem of computing the ${\text{ nc-rank }}$ of a matrix by testing fullness of a smaller matrix with polynomial entries.

Lemma A.3

(Reduction to Fullness Testing) Let $M \in \mathbb {F}\langle {\mathbf {x}}\rangle ^{m \times n}$ be any matrix. In addition, let $U = (u_{ij})$ and $V = (v_{ij})$ be generic matrices in new, non-commuting variables $u_{ij}$ and $v_{ij}$, of dimensions $r \times m$ and $n \times r$, respectively. Then, ${\text{ nc-rank }}(M) \ge r$ iff the matrix UMV is full.

Proof

Since ${\text{ nc-rank }}(M) \ge r$, there exists an $r \times r$ minor of M of full rank. Let Q be such a minor of M. W.l.o.g.,²⁸ we can assume that Q is the $[r] \times [r]$ principal minor of M. Hence, we have that

$$\begin{aligned} UMV = \begin{pmatrix} U_1&\quad U_2 \end{pmatrix} \begin{pmatrix} Q &{}\quad M_2 \\ M_3 &{}\quad M_4 \end{pmatrix} \begin{pmatrix} V_1 \\ V_2 \end{pmatrix}, \end{aligned}$$

where $U_1$ and $V_1$ are $r \times r$ matrices and the others are matrices with the proper dimensions.

Letting $U' = \begin{pmatrix} I_r&0 \end{pmatrix}$ and $V' = \begin{pmatrix} I_r \\ 0 \end{pmatrix}$, the equality above becomes:

$$\begin{aligned} U'MV' = Q. \end{aligned}$$

As

$$\begin{aligned} r \ge {\text{ nc-rank }}(UMV) \ge {\text{ nc-rank }}(U'MV') = {\text{ nc-rank }}(Q) = r, \end{aligned}$$

we obtain that UMV is full, as we wanted. Notice that the second inequality comes from the fact that rank does not increase after restrictions of the new variables. $\square$

Notice that we do not know the rank ${\text{ nc-rank }}(M)$ a priori. Therefore, our algorithm will try all possible values of $r \in [n]$ and output the maximum value of r for which we find a full matrix.

For each $r \times r$ matrix UMV, we can use the effective Higman’s trick to convert UMV into a $s \times s$ matrix with linear entries. With this matrix, we can use the truncated Gurvits’ algorithm to check whether the matrix we just obtained is full. Since we have this test, we will be able to output the correct rank. Algorithm 5 is the precise formulation of the procedure just described.

Theorem A.4

Let $M \in \mathbb {Q}\langle {\mathbf {x}}\rangle ^{m \times n}$ be s.t. each entry of M is a polynomial computed by a formula of size bounded by s and bit complexity bounded by b. There exists a deterministic algorithm that finds the non-commutative rank of M in time ${\text{ poly }}(m, n, s, b)$.

Proof

To prove this theorem, it is enough to show that Algorithm 5 is correct and it runs with the desired runtime.

Without loss of generality, we can assume that $n \le m$. Therefore we have that ${\text{ nc-rank }}(M) \le n$. By Lemma A.3, if $r \le {\text{ nc-rank }}(M)$, then matrix $M_r$ will be of full rank (and therefore will not have a shrunk subspace, by Theorem 1.4). Since $M_r = U_r M V_r$, from the formulas computing the entries of M, we obtain formulas of size at most 2smn computing the entries of $M_r$. Moreover, the bit complexities of these formulas will still be bounded by b, as multiplication by generic matrices do not mix any of the polynomials of M.

By Proposition A.2 and the fact that the size of the formulas computing the entries of $M_r$ are bounded by 2smn, we have that $N_r$ is a linear matrix of dimensions $(k+r) \times (k+r)$, where $k \le 2s(mn)^2$ and the bit complexity of the coefficients bounded by b. Moreover, $N_r = P (M_r \oplus I_{k}) Q$ implies that $N_r$ is full if, and only if, $M_r$ is full, which is true if, and only if, ${\text{ nc-rank }}(M) \ge r$.

Now, by Theorem 1.1, we have a deterministic polynomial time algorithm to determine whether $N_r$ is full rank. If $r \le {\text{ nc-rank }}(M)$, $N_r$ will be full, and the maximum such r will be exactly when $r = {\text{ nc-rank }}(M)$. Therefore, by outputting the maximum r for which $N_r$ we compute ${\text{ nc-rank }}(M)$. This proves that our algorithm is correct. Notice that the runtime is polynomial in the input size, as we perform at most n applications of the Higman trick and of Algorithm $G'$. This completes the proof. $\square$

1.3 The Quantum Reduction

Here we present a different reduction from computing non-commutative rank to fullness testing from a quantum viewpoint. We will only work with square matrices though. As we saw, by Higman’s trick, we can assume the matrices to be linear. So we are given a matrix $L = \sum _{i=1}^m x_i A_i \in M_n( \mathbb {F}\langle {\mathbf {x}}\rangle )$. A combination of Theorems 1.4 and 1.17 shows that ${\text{ nc-rank }}(L) \le r$ iff the operator defined by $A_1,\ldots ,A_m$ is $n-r$-rank-decreasing. So we just want to check whether a completely positive operator is c-rank-decreasing, and we will do this by using an algorithm for checking if an operator is rank-decreasing as a black box using the following lemma:

Lemma A.5

Let $T : M_n(\mathbb {C}) \rightarrow M_n(\mathbb {C})$ be a completely positive operator. Define an operator $\overline{T} : M_{n+c-1}(\mathbb {C}) \rightarrow M_{n+c-1}(\mathbb {C})$ as follows:

$$\begin{aligned} \overline{T} \left( \begin{bmatrix} X_{1,1}&\quad X_{1,2} \\ X_{2,1}&\quad X_{2,2} \end{bmatrix} \right) = \begin{bmatrix} T(X_{1,1}) + \text {tr}(X_{2,2})I_n&\quad 0 \\ 0&\quad \text {tr}(X_{1,1})I_{c-1} \end{bmatrix} \end{aligned}$$

Here $X_{1,1}$, $X_{1,2}$, $X_{2,1}$, $X_{2,2}$ are $n \times n$, $n \times c-1$, $c-1 \times n$, $c-1 \times c-1$ matrices, respectively. Then $\overline{T}$ is completely positive and T is c-rank-decreasing iff $\overline{T}$ is rank-decreasing. Note that we are considering $c \le n$.

Proof

A well known characterization due to Choi [12] states that $\overline{T}$ is completely positive iff $\sum _{i,j = 1}^{n+c-1} E_{i,j} \otimes \overline{T}(E_{i,j})$ is psd. Here $E_{i,j}$ is the matrix with 1 at i, j position and 0 everywhere else. Now

$$\begin{aligned}&\overline{T}(E_{i,j}) \\&\quad = {\left\{ \begin{array}{ll} \begin{bmatrix} T(E_{i,j}) &{}\quad 0 \\ 0 &{}\quad I_{c-1} \end{bmatrix} &{}\quad 1 \le i=j \le n \\ \\ \begin{bmatrix} T(E_{i,j}) &{}\quad 0 \\ 0 &{}\quad 0 \end{bmatrix} &{} 1 \le i, j \le n, i \ne j \\ \\ \begin{bmatrix} 0 &{}\quad 0 \\ 0 &{}\quad 0 \end{bmatrix} &{}\quad 1 \le i \le n, n+1 \le j \le n+c-1 \, \text {or} \, n+1 \le i \le n+c-1, 1 \le j \le n \\ \\ \begin{bmatrix} I_n &{}\quad 0 \\ 0 &{}\quad 0 \end{bmatrix} &{} n+1 \le i=j \le n+c-1 \\ \\ \begin{bmatrix} 0 &{}\quad 0 \\ 0 &{}\quad 0 \end{bmatrix} &{} n+1 \le i, j \le n+c-1, i \ne j \\ \end{array}\right. } \end{aligned}$$

From here, it is easy to verify that $\sum _{i,j = 1}^{n+c-1} E_{i,j} \otimes \overline{T}(E_{i,j})$ is psd given that $\sum _{i,j=1}^n E_{i,j} \otimes T(E_{i,j})$ is psd. Now suppose that $\overline{T}$ is rank-decreasing. This can only happen if $X_{1,1} = 0$ or $X_{2,2} = 0$, otherwise

$$\begin{aligned} \overline{T} \left( \begin{bmatrix} X_{1,1}&\quad X_{1,2} \\ X_{2,1}&\quad X_{2,2} \end{bmatrix} \right) = \begin{bmatrix} T(X_{1,1}) + \text {tr}(X_{2,2})I_n&0 \\ 0&\quad \text {tr}(X_{1,1})I_{c-1} \end{bmatrix} \end{aligned}$$

is full rank. If $X_{1,1} = 0$, then

$$\begin{aligned} \begin{bmatrix} 0&X_{1,2} \\ X_{2,1}&X_{2,2} \end{bmatrix} \end{aligned}$$

can be psd (and Hermitian) only if $X_{1,2} = X_{2,1} = 0$. In this case a $c-1$ ranked matrix is mapped to rank n matrix. So $X_{2,2}$ has to be zero. Then again by the psd condition $X_{1,2} = X_{2,1} = 0$. So

$$\begin{aligned} \overline{T} \left( \begin{bmatrix} X_{1,1}&\quad 0 \\ 0&\quad 0 \end{bmatrix} \right) = \begin{bmatrix} T(X_{1,1})&\quad 0 \\ 0&\quad \text {tr}(X_{1,1})I_{c-1} \end{bmatrix} \end{aligned}$$

and $X_{1,1} \ne 0$ and

$$\begin{aligned} \text {Rank} \left( \begin{bmatrix} X_{1,1}&\quad 0 \\ 0&\quad 0 \end{bmatrix} \right) > \text {Rank} \left( \begin{bmatrix} T(X_{1,1})&\quad 0 \\ 0&\quad \text {tr}(X_{1,1})I_{c-1} \end{bmatrix} \right) \end{aligned}$$

Hence $\text {Rank}(T(X_{1,1})) \le \text {Rank}(X_{1,1}) - c$. This proves one direction. Now suppose that T is c-rank-decreasing and $\text {Rank}(T(X)) \le \text {Rank}(X)-c$, then

$$\begin{aligned} \text {Rank} \left( \overline{T} \left( \begin{bmatrix} X&\quad 0 \\ 0&\quad 0 \end{bmatrix} \right) \right) = \text {Rank} \left( \begin{bmatrix} T(X)&\quad 0 \\ 0&\quad \text {tr}(X) I_{c-1} \end{bmatrix} \right) < \text {Rank} \left( \begin{bmatrix} X&\quad 0 \\ 0&\quad 0 \end{bmatrix} \right) \end{aligned}$$

This proves the lemma. $\square$

Remark A.6

This seems to be the “quantum” analogue of obtaining a maximum matching oracle based on a perfect matching oracle: add c-1 dummy vertices to both sides of the bipartite graph and connect them to everything. Then the new graph has a perfect matching iff the original graph had a matching of size $\ge n - c + 1$.

Remark A.7

Here we didn’t specify a set of Kraus operators for the operator $\overline{T}$ which seem to be needed to run Algorithms 1 and 2 but Kraus operators can be obtained by looking at the eigenvectors of $\sum _{i,j=1}^{n+c-1} E_{i,j} \otimes \overline{T}(E_{i,j})$. Alternatively Algorithms 1 and 2 can also be interpreted as acting directly on the Choi-Jamiolkowski state of $\overline{T}$, i.e., $\sum _{i,j=1}^{n+c-1} E_{i,j} \otimes \overline{T}(E_{i,j})$.