Generalized Macdonald Functions on Fock Tensor Spaces and Duality Formula for Changing Preferred Direction

Abstract

An explicit formula is obtained for the generalized Macdonald functions on the N-fold Fock tensor spaces, calculating a certain matrix element of a composition of several screened vertex operators. As an application, we prove the factorization property of the arbitrary matrix elements of the multi-valent intertwining operator (or refined topological vertex operator) associated with the Ding–Iohara–Miki algebra (DIM algebra) with respect to the generalized Macdonald functions, which was conjectured by Awata, Feigin, Hoshino, Kanai, Yanagida and one of the authors. Our proof is based on the combinatorial and analytic properties of the asymptotic eigenfunctions of the ordinary Macdonald operator of A-type, and the Euler transformation formula for Kajihara and Noumi’s multiple basic hypergeometric series. That factorization formula provides us with a reasonable algebraic description of the 5D (K-theoretic) Alday-Gaiotto–Tachikawa (AGT) correspondence, and the interpretation of the invariance under the preferred direction from the point of view of the $SL(2,{\mathbb {Z}})$ duality of the DIM algebra.

Appendices

Appendix A. Construction of Macdonald Symmetric Functions in terms of Topological Vertex

We can obtain Macdonald functions as matrix elements of some compositions of the intertwining operators. The similar derivation and its supersymmetric version are given in [Z]. Set

(A.1)

where the overall factor is chosen for the later convenience. Though the prefactor consisting of the ${\mathcal {G}}$-factors gives zeros under the restriction of $\varvec{v}$, but the ${\mathcal {G}}$-factors which appear from the normal orderings among $\Phi$’s and $\Phi ^*$’s in $\mathcal {T}^V$, cancel those zeros, and thus the operator is well-defined. By operator products (C.26) and Lemma C.1, it can be seen that the Young diagrams are restricted to only one row when gluing intertwiners over the vertical representations (Fig. 2), i.e.,

(A.2)

where we put $m_0 = 0$. Here, is introduced in Notation 5.7. Then, we have $\widetilde{{\mathcal {T}}}_1(x)=\Phi ^{(0)}(t^{-1}x)$. Note that the operators $\Phi ^{(k)}(x)\,\,(k=0,\dots ,N-1)$ in this section slightly differ from those in Sec. 3.2. In this section, $\Phi ^{(k)}(x)$ is a map $\mathcal {F}_{t^{-\delta _{k+1}}\cdot {\tilde{\varvec{u}}}} \rightarrow \mathcal {F}_{\varvec{u}}$ with ${\tilde{\varvec{u}}}=(\gamma ^{-1}u_1, \ldots , \gamma ^{-1}u_N)$, that is, the spectral parameter differs by the factor $\gamma ^{-1}$.

Fig. 2

The operator $\widetilde{{\mathcal {T}}}_i(x)$. ($\Longrightarrow$ stands for a simplification of the diagram for convenience)

We obtain the expression of Macdonald functions in terms of intertwiners of the DIM algebra. The following proposition says the vacuum expectation value of Fig. 3 gives the Macdonald function of the A type.

Fig. 3

The diagram for the Macdonald function $p_{N}(\varvec{x};\varvec{u}|q,q/t)$

Proposition A.1

(A.3)

Proof

The operators $\widehat{\Phi }_{(m)}(z)$ and $\widehat{\Phi }^*_{(m)}(z)$ can be decomposed as

$$\begin{aligned} \widehat{\Phi }_{(m)}(z)=:\widehat{\Phi }_{\emptyset }(t^{-1}z) {\mathcal {A}}(q^mz):, \quad \widehat{\Phi }^*_{(m)}(z)=:\widehat{\Phi }_{\emptyset }^*(t^{-1}z) {\mathcal {A}}^*(q^mz):, \end{aligned}$$

(A.4)

where

$$\begin{aligned}&{\mathcal {A}}(z)= \exp \left( -\sum _{n>0}\frac{1-t^{-n}}{n(1-q^n)} a_{-n}z^n\right) \exp \left( \sum _{n>0}\frac{1-t^n}{n(1-q^{-n})} a_{n} z^{-n}\right) , \end{aligned}$$

(A.5)

$$\begin{aligned}&{\mathcal {A}}^*(z)= \exp \left( \sum _{n>0}\frac{1-t^{-n}}{n(1-q^n)}\gamma ^n a_{-n}z^n\right) \exp \left( -\sum _{n>0}\frac{1-t^n}{n(1-q^{-n})} \gamma ^{n}a_{n} z^{-n}\right) . \end{aligned}$$

(A.6)

Then the tensor product of ${\mathcal {A}}(z)$ and ${\mathcal {A}}^*(z)$ corresponds to the screening current (Definition 3.10):

$$\begin{aligned} {\mathcal {A}}^*(z)\otimes {\mathcal {A}}(z)=\phi ^{sc}(\gamma ^{-1}t^{-1}z). \end{aligned}$$

(A.7)

Therefore, we have

$$\begin{aligned} \widetilde{{\mathcal {T}}}_i(\varvec{u}; x) =&\left( \frac{(q/t;q)_{\infty }}{(q;q)_{\infty }}\right) ^{i-1} \times \sum _{0\le m_1\le m_2\le \cdots \le m_{i-1} <\infty } \Phi ^{(0)}(t^{-1}x) \widetilde{S}^{(1)}(q^{m_1}x) \nonumber \\&\quad \cdots \widetilde{S}^{(i-1)}(q^{m_{i-1}}x) \prod _{k=1}^{i-1}(u_{k+1}/u_{k})^{m_k}. \end{aligned}$$

(A.8)

Here, we put

$$\begin{aligned} \widetilde{S}^{(k)} (z)=S^{(k)}(\gamma ^{-2k}t^{-1} z). \end{aligned}$$

(A.9)

As in the proof of Proposition 3.12 (Appendix D.1), $X^{(1)}(z)$ commutes with $\widetilde{S}^{(k)}(w)$ up to q-difference:

$$\begin{aligned} \left[ X^{(1)}(z),\widetilde{S}^{(k)}(w)\right] =(t-1)(u_{k+1}T_{q,w}-u_k) \left( \delta \left( \frac{\gamma ^{-2k}w}{qz} \right) :\Lambda ^{(k)}(\gamma ^{-2k }w/q)\widetilde{S}^{(k)}(w):\right) . \end{aligned}$$

(A.10)

Further, by the property of the operator products

$$\begin{aligned} \widetilde{S}^{(k-1)}(x)\Lambda ^{(k)}(\gamma ^{-2k }x/q) =\Phi ^{(0)}(t^{-1}x) \Lambda ^{(1)}(\gamma ^{-2}x/q)=0, \end{aligned}$$

(A.11)

we obtain

$$\begin{aligned}&\Phi ^{(0)}(t^{-1}x) \cdot \left[ X^{(1)}_{0}, \sum _{m=0}^{\infty } \widetilde{S}^{(1)}(q^{m}x)(u_{2}/u_{1})^{m}\right] =0, \end{aligned}$$

(A.12)

$$\begin{aligned}&\widetilde{S}^{(k-1)}(x)\cdot \left[ X^{(1)}_{0}, \sum _{m=0}^{\infty } \widetilde{S}^{(k)}(q^{m}x)(u_{k+1}/u_{k})^{m}\right] =0 \quad (k\ge 2). \end{aligned}$$

(A.13)

This leads to

$$\begin{aligned}&X^{(1)}(z) \widetilde{{\mathcal {T}}}_i(\varvec{u};w) - \gamma \frac{1-qz/tw}{1-z/w} \widetilde{{\mathcal {T}}}_i(\varvec{u};w)X^{(1)}(z) \nonumber \\&\quad =u_i(1-t^{-1}) \widetilde{{\mathcal {T}}_i}(\varvec{u};qw)\Psi ^+(t^{-1}w) \delta (w/z). \end{aligned}$$

(A.14)

By this relation, we get

(A.15)

Thus, the considered matrix elements are eigenfunctions of the difference operator $D_{N}^1$:

(A.16)

$\square$

Remark A.2

By using the expression (A.8), it is shown that the operator $\widetilde{{\mathcal {T}}}_i(x)$ also represents the screened vertex operator $\Phi ^{(i-1)}(x)$. Whereas $\widetilde{{\mathcal {T}}}_i(x)$ is written by a sort of Jackson integrals, $\Phi ^{(i-1)}(x)$ is introduced by counter integrals. In order to justify the equivalence of the two expressions, see the following argument.

We first note the following deformation of the formal series,

$$\begin{aligned} \begin{aligned} \text {(Jackson integral)}\quad \sum _{m\in {\mathbb {Z}}_{\ge 0}} \delta (q^m z) \alpha ^m&=\sum _{m\in {\mathbb {Z}}_{\ge 0}, k \in {\mathbb {Z}}} (q^m z)^k \alpha ^m \\&= \sum _{k\in {\mathbb {Z}}} \frac{1}{1-q^k \alpha } z^k \\&= \frac{\theta _q(\alpha )}{(q;q)_\infty (q^{-1};q)_\infty }\frac{\theta _q (\alpha z )}{\theta _q (z)}\quad \text {(Contour integral)}. \end{aligned} \end{aligned}$$

(A.17)

Of course, this series does not converge for arbitrary q and $\alpha$. However, by the following reason, we can choose some domains for q and $\alpha$ so that the matrix elements of $\widetilde{{\mathcal {T}}}_i(x)$ converge there, and thus they are identical to those of $\Phi ^{(i-1)}(x)$. We discuss the example of the $i=2$ case. The matrix elements of $\widetilde{{\mathcal {T}}}_2(x)$ are of the form,

(A.18)

where $\alpha = u_2/u_1$, and $\sim$ means the both sides are equal up to some overall factors. The factor is some finite sum of the monomial of the form $x^a w^b$ with the maximums and minimums of a’s and b’s depending on $\varvec{\lambda }$ and $\varvec{\mu }$. Then, by taking the constant term in w in the above, the running index k is fixed as $k = n + b \ge b$. Therefore, when we choose the q and $\alpha$ so that

$$\begin{aligned} |\alpha q^{\mathrm {min}(b)}| \ll 1, \end{aligned}$$

(A.19)

the deformation (A.17) is justified by putting $k< \mathrm {min} (b)$ terms zero, and thus

(A.20)

This discussion can extend to the general i case, and the equivalence between the Jackson integral and contour integral is justified.

Appendix B. Basic Facts for Ordinary Macdonald Functions

In this section, we give the definition and basic facts for ordinary Macdonald functions. Let $\Lambda _n={\mathbb {C}}[x_1,\ldots ,x_n]^{\mathfrak {S}_n}$ be the ring of symmetric polynomials, and $\Lambda =\lim _{\leftarrow }\Lambda _n$ be the projective limit in the category of graded rings, i.e., the ring of symmetric functions. $m_{\lambda }$ denotes the monomial symmetric function. We denote the power sum symmetric functions by $\mathsf {p}_n=\mathsf {p}_n(x)=\sum _{i \ge 1} x^n_i$. For a partition $\lambda$, set $\mathsf {p}_{\lambda }=\prod _{i\ge 1} \mathsf {p}_{\lambda _{i}}$. Macdonald functions are defined as orthogonal functions with respect to the following scalar product.

Definition B.1

Define the bilinear form $\langle -,- \rangle _{q,t}:\Lambda \otimes \Lambda \rightarrow {\mathbb {C}}$ by

$$\begin{aligned} \langle \mathsf {p}_{\lambda }, \mathsf {p}_{\mu } \rangle _{q,t} =\delta _{\lambda , \mu }z_{\lambda } \prod _{i=1}^{\ell (\lambda )} \frac{1-q^{\lambda _i}}{1-t^{\lambda _i}}, \quad z_{\lambda }:=\prod _{i\ge 1}i^{m_i}\cdot m_i!, \end{aligned}$$

(B.1)

where $m_i$ is the number of entries in $\lambda$ equal to i.

Fact B.2

([Ma]) There exists an unique function $P_{\lambda }\in \Lambda$ such that

$$\begin{aligned}&P_{\lambda }=m_{\lambda }+\sum _{\mu < \lambda } \alpha _{\lambda ,\mu } m_{\mu }\quad (\alpha _{\lambda ,\mu } \in {\mathbb {C}});\end{aligned}$$

(B.2)

$$\begin{aligned}&\langle P_{\lambda }, P_{\mu } \rangle _{q,t}=0 \quad (\lambda \ne \mu ). \end{aligned}$$

(B.3)

The symmetric functions $P_{\lambda }$ are called Macdonald symmetric functions. It is known that

$$\begin{aligned} \langle P_{\lambda }, P_{\lambda } \rangle _{q,t}=c'_{\lambda }/c_{\lambda }. \end{aligned}$$

(B.4)

Here, $c_{\lambda }$ and $c'_{\lambda }$ are defined in (3.14). Set $Q_{\lambda }:=\langle P_{\lambda }, P_{\lambda } \rangle _{q,t}^{-1}\cdot P_{\lambda }$, so that $\langle P_{\lambda }, Q_{\lambda } \rangle _{q,t}=1$. In this paper, we regard power sum symmetric functions as variables of Macdonald functions and write $P_{\lambda }=P_{\lambda }(\mathsf {p}_n)$, $Q_{\lambda }=Q_{\lambda }(\mathsf {p}_n)$. These are abbreviation for $P_{\lambda }(\mathsf {p}_1,\mathsf {p}_2,\ldots )$ and $Q_{\lambda }(\mathsf {p}_1,\mathsf {p}_2,\ldots )$. We often substitute the generators $a_{n}$ in the Heisenberg algebra into Macdonald functions as $P_{\lambda }(a_{-n})$. Note that this substitution preserves the properties of Macdonald functions as follows. The space of symmetric functions $\Lambda$ and the Fock space ${\mathcal {F}}$ are isomorphic as graded vector spaces, and they can be identified by

(B.5)

Further, the bilinear form on the Fock spaces preserves the structure of the scalar product $\langle -, - \rangle _{q,t}$ under this identification, i.e.,

(B.6)

We describe some facts for Macdonald functions that are used in this paper.

1.1 Kernel function

The following function $\Pi (x,y;q,t)$ is called the kernel function:

$$\begin{aligned} \Pi (x,y;q,t):=\prod _{n\ge 1}\exp \left( \frac{1-t^n}{1-q^n} \mathsf {p}_n(x)\mathsf {p}_n(y) \right) . \end{aligned}$$

(B.7)

This function $\Pi (x,y;q,t)$ can be expanded in terms of dual bases parametrized by partitions as follows.

Fact B.3

([Ma]) Let $u_{\lambda }$, $v_{\lambda } \in \Lambda$ be homogeneous symmetric functions of level $|\lambda |$ and $\{u_{\lambda }\}$, $\{v_{\lambda }\}$ form ${\mathbb {C}}$-bases on $\Lambda$. Then, the followings are equivalent:

$$\begin{aligned}&\cdot \; \langle u_{\lambda }, v_{\lambda } \rangle _{q,t} =\delta _{\lambda , \mu } \quad \text{ for } \text{ al } {\lambda , \mu }; \end{aligned}$$

(B.8)

$$\begin{aligned}&\cdot \; \Pi (x,y;q,t) =\sum _{\lambda }u_{\lambda }(x) v_{\lambda }(y). \end{aligned}$$

(B.9)

Note that especially, we have

$$\begin{aligned} \Pi (x,y;q,t)=\sum _{\lambda } P_{\lambda }(\mathsf {p}_n(x)) Q_{\lambda }(\mathsf {p}_n(y)). \end{aligned}$$

(B.10)

1.2 Pieri formula

Define the symmetric function $g_r\in \Lambda$ to be the expansion coefficient of $y^r$ in the following series:

$$\begin{aligned} \exp \left( \sum _{n\ge 1} \frac{1}{n}\frac{1-t^n}{1-q^n} \mathsf {p}_n y^n\right) =\sum _{r\ge 0} g_r \, y^r. \end{aligned}$$

(B.11)

For a partition $\lambda$ and a coordinate s, we write

$$\begin{aligned} b_{\lambda }(s)= {\left\{ \begin{array}{ll} \dfrac{1-q^{a_{\lambda }(s)}t^{l_{\lambda }(s)+1}}{1-q^{a_{\lambda }(s)+1}t^{l_{\lambda }(s)}}, &{} s \in \lambda ; \\ 1, &{} \mathrm {otherwise}. \end{array}\right. } \end{aligned}$$

(B.12)

Fact B.4

([Ma]) We have the Pieri rules:

$$\begin{aligned} g_rQ_{\mu }=\sum _{\lambda }\prod _{s\in R_{\lambda /\mu }- C_{\lambda /\mu }}\frac{b_{\mu }(s)}{b_{\lambda }(s)} \cdot Q_{\lambda }. \end{aligned}$$

(B.13)

Here, the summation is over the partitions $\lambda$ such that $\lambda /\mu$ is a horizontal r-strip, i.e., $\lambda /\mu$ has at most one box in each column. $R_{\lambda /\mu }$ (resp. $C_{\lambda /\mu }$) is the union of the rows (resp. columns) that intersect $\lambda -\mu$.

Note that by this formula, we can see that $Q_{\mu }$ ($\mu \ngeq \lambda$) does not appear in the expansion of the product $\prod _{i \ge 1} g_{\lambda _i}$ in the basis of Macdonald functions.

1.3 Another scalar product

There is another scalar product for which Macdonald functions $P_{\lambda }$’s are pairwise orthogonal. Let $L_r={\mathbb {C}}[x_{1}^{\pm 1},\ldots , x_{r}^{\pm }]$ be the ${\mathbb {C}}$-algebra of Laurent polynomials in r variables. For $f(x_1,\ldots , x_r) \in L_r$, we put ${{\bar{f}}}= f(x_1^{-1},\ldots , x_r^{-1})$.

Definition B.5

The scalar product $\langle -,- \rangle '_r$ on $L_r$ is defined by

$$\begin{aligned} \langle f,g \rangle '_r:=\oint \prod _{i=1}^r\frac{dx_i}{2 \pi \sqrt{-1}} f {{\bar{g}}} \Delta (x), \end{aligned}$$

(B.14)

where

$$\begin{aligned} \Delta (x)&:=\prod _{ i<j} \frac{(x_i/ x_j;q)_{\infty }(x_j/ x_i;q)_{\infty }}{(tx_i/x_j;q)_{\infty }(tx_j/x_i;q)_{\infty }}. \end{aligned}$$

(B.15)

Fact B.6

([Ma]) Let $P_{\lambda }^{(r)}=P_{\lambda } \big |_{x_i \rightarrow 0\, (i>r)}, \, Q_{\lambda }^{(r)}=Q_{\lambda } \big |_{x_i \rightarrow 0\, (i>r)} \in \Lambda _r$ be the Macdonald symmetric polynomials in r variables. Then

$$\begin{aligned} \langle P_{\lambda }^{(r)},Q_{\mu }^{(r)} \rangle '_r =\delta _{\lambda , \mu } \langle 1,1 \rangle '_r \prod _{(i,j) \in \lambda }\frac{1-q^{j-1}t^{r-i+1} }{1-q^{j}t^{r-i} }, \end{aligned}$$

(B.16)

where $\ell (\lambda ), \ell (\mu ) \le r$.

Appendix C. Some Useful Formulas

1.1 C.1 List of operator products

In this subsection, we list some formulas for the normal ordering among the various operators appeared in the main text. We have

$$\begin{aligned}&\Lambda ^{(i)}(z) S^{(i)}(w) = \frac{1-t^2w/qz}{1-tw/qz}:\Lambda ^{(i)}(z) S^{(i)}(w):\,, \end{aligned}$$

(C.1)

$$\begin{aligned}&\Lambda ^{(i+1)}(z) S^{(i)}(w) = \frac{1-w/z}{1-tw/z}:\Lambda ^{(i+1)}(z) S^{(i)}(w):\,,\end{aligned}$$

(C.2)

$$\begin{aligned}&\Lambda ^{(j)}(z) S^{(i)}(w) = :\Lambda ^{(j)}(z) S^{(i)}(w): \quad \text {for} \quad j<i\quad \text {and}\quad j>i+1,\end{aligned}$$

(C.3)

$$\begin{aligned}&S^{(i)}(w)\Lambda ^{(i)}(z) = \frac{1-qz/t^2w}{1-qz/tw}:S^{(i)}(w)\Lambda ^{(i)}(z):\,,\end{aligned}$$

(C.4)

$$\begin{aligned}&S^{(i)}(w)\Lambda ^{(i+1)}(z) = \frac{1-z/w}{1-z/tw}:S^{(i)}(w)\Lambda ^{(i+1)}(z):\,,\end{aligned}$$

(C.5)

$$\begin{aligned}&S^{(i)}(w)\Lambda ^{(j)}(z) = :S^{(i)}(w)\Lambda ^{(j)}(z): \quad \text {for} \quad j<i\quad \text {and}\quad j>i+1, \end{aligned}$$

(C.6)

$$\begin{aligned}&\Phi ^{(0)}(z) S^{(1)}(w)=\frac{(t w/ z ; q)_\infty }{(w/z ; q)_\infty }:\Phi ^{(0)}(z) S^{(1)}(w):, \end{aligned}$$

(C.7)

$$\begin{aligned}&\Phi ^{(0)}(z) S^{(i)}(w)=:\Phi ^{(0)}(z) S^{(i)}(w): \quad (i\ge 2), \end{aligned}$$

(C.8)

$$\begin{aligned}&S^{(1)}(z)\Phi ^{(0)}(w)=\frac{(qw/ z ; q)_\infty }{(q w/t z ; q)_\infty }:S^{(1)}(z)\Phi ^{(0)}(w):,\end{aligned}$$

(C.9)

$$\begin{aligned}&S^{(i)}(z)\Phi ^{(0)}(w)=:S^{(i)}(z)\Phi ^{(0)}(w): \quad (i\ge 2), \end{aligned}$$

(C.10)

$$\begin{aligned}&\Phi ^{(0)}(z)\Phi ^{(0)}(w)=\frac{(qw/tz;q)_{\infty }}{(tw/z;q)_{\infty }} :\Phi ^{(0)}(z)\Phi ^{(0)}(w):, \end{aligned}$$

(C.11)

$$\begin{aligned}&S^{(i)}(z)S^{(i)}(w) = (1- w/z) \frac{(q w/ t z ; q)_\infty }{(t w/z ; q)_\infty }:S^{(i)}(z)S^{(i)}(w):\,, \end{aligned}$$

(C.12)

$$\begin{aligned}&S^{(i)}(z)S^{(i+1)}(w) = \frac{(t w/ z ; q)_\infty }{( w/z ; q)_\infty }:S^{(i)}(z)S^{(i+1)}(w):, \end{aligned}$$

(C.13)

$$\begin{aligned}&S^{(i+1)}(z)S^{(i)}(w)= \frac{(qw/ z ; q)_\infty }{(q w/t z ; q)_\infty }:S^{(i+1)}(z)S^{(i)}(w): \quad (\forall i),\end{aligned}$$

(C.14)

$$\begin{aligned}&S^{(i)}(z)S^{(j)}(w) = :S^{(i)}(z)S^{(j)}(w): \quad \text {for} \quad |i-j|>2, \end{aligned}$$

(C.15)

$$\begin{aligned}&\Lambda ^{(1)}(z) \Phi ^{(0)}(x) = \frac{1-x/z}{1-tx/z}:\Lambda ^{(1)}(z) \Phi ^{(0)}(x) :\,,\end{aligned}$$

(C.16)

$$\begin{aligned}&\Phi ^{(0)}(x) \Lambda ^{(1)}(z) = \frac{1-z/x}{1-qz/t^2x}:\Phi ^{(0)}(x) \Lambda ^{(1)}(z):\,, \end{aligned}$$

(C.17)

$$\begin{aligned}&\Lambda ^{(i)}(z) \Phi ^{(0)}(x) = : \Lambda ^{(i)}(z) \Phi ^{(0)}(x):\,, \end{aligned}$$

(C.18)

$$\begin{aligned}&\Phi ^{(0)}(x) \Lambda ^{(i)}(z) = \frac{1-z/tx}{1-qz/t^2x}:\Phi ^{(0)}(x) \Lambda ^{(i)}(z):\quad (i>1), \end{aligned}$$

(C.19)

$$\begin{aligned}&\Psi ^+(z)S^{(i)}(w)=S^{(i)}(w)\Psi ^+(z)=:\Psi ^+(z)S^{(i)}(w): \quad (\forall i), \end{aligned}$$

(C.20)

$$\begin{aligned}&\Psi ^+(z)\Phi ^{(0)}(w) = \frac{1-tw/qz}{1-w/z}:\Psi ^+(z)\Phi ^{(0)}(w):, \end{aligned}$$

(C.21)

$$\begin{aligned}&A_{(s)}(x)\Lambda ^{(i)}(z) = \prod _{k=1}^{r-1}\frac{1-t^{-k}z/x}{1-t^{-k-1}qz/x} :\Lambda ^{(i)}(z) A_{(s)}(x):, \end{aligned}$$

(C.22)

$$\begin{aligned}&A^{(r)}(x) \Phi ^{(0)}(y)=\prod _{k=0}^{r-2}\frac{(t^{-k}qy/tx;q)_{\infty }}{(t^{-k}y/x;q)_{\infty }}:A^{(r)}(x) \Phi ^{(0)}(y):, \end{aligned}$$

(C.23)

$$\begin{aligned}&A^{(r)}(x) S^{(i)}(y)=:A^{(r)}(x) S^{(i)}(y):, \end{aligned}$$

(C.24)

and with ${\mathcal {G}}(z) = \prod _{i,j=0}^\infty (1 - z q^i t^{-j})$,

$$\begin{aligned}&\Phi _{\lambda }(v_i) \Phi ^*_{\mu }(u_j) = {\mathcal {G}}(u_j/\gamma v_i)^{-1} N_{\mu \lambda }(u_j/\gamma v_i):\Phi _{\lambda }(v_i) \Phi ^*_{\mu }(u_j):, \end{aligned}$$

(C.25)

$$\begin{aligned}&\Phi ^*_{\mu }(u_j)\Phi _{\lambda }(v_i) = {\mathcal {G}}(v_i/\gamma u_j)^{-1} N_{ \lambda \mu }(v_i/\gamma u_j):\Phi ^*_{\mu }(u_j)\Phi _{\lambda }(v_i):, \end{aligned}$$

(C.26)

$$\begin{aligned}&\Phi _{\lambda ^{(i)}}(v_i) \Phi _{\lambda ^{(j)}}(v_j) = \frac{{\mathcal {G}}(v_j/\gamma ^2 v_i)}{N_{\lambda ^{(j)}\lambda ^{(i)}}(v_j/\gamma ^2 v_i)}:\Phi _{\lambda ^{(i)}}(v_i) \Phi _{\lambda ^{(j)}}(v_j):, \end{aligned}$$

(C.27)

$$\begin{aligned}&\Phi ^*_{\mu ^{(i)}}(u_i) \Phi ^*_{\mu ^{(j)}}(u_j) = \frac{{\mathcal {G}}(u_j/u_i)}{N_{\mu ^{(j)}\mu ^{(i)}}(u_j/u_i)}:\Phi ^*_{\mu ^{(i)}}(u_i) \Phi ^*_{\mu ^{(j)}}(u_j): . \end{aligned}$$

(C.28)

1.2 C.2 On Nekrasov factors

For a partitions $\lambda$ and non-negative integers $r, s \in {\mathbb {Z}}_{\ge 0}$, define ${\mathsf {B}}_{r,s}(\lambda )$ to be the partition obtained by removing the 1st to s-th rows and the 1st to r-th columns, i.e.,

$$\begin{aligned} {\mathsf {B}}_{r,s}(\lambda ) :=({\mathsf {P}}(\lambda _{s+i}-r) )_{i\ge 1}, \quad {\mathsf {P}}(n)= {\left\{ \begin{array}{ll} n, \quad n\ge 0;\\ 0, \quad n<0. \end{array}\right. } \end{aligned}$$

(C.29)

For example, if $\lambda =(5,5,4,4,4,1,1)$, then ${\mathsf {B}}_{2,1}(\lambda )=(3,2,2,2)$. We have the following vanishing condition for the Nekrasov factor.

Lemma C.1

If $m\ge 0$ and $n\le 0$, then

$$\begin{aligned} N_{\lambda ,\mu }(q^n t^m) \ne 0\quad \Longleftrightarrow \quad \mu \supset {\mathsf {B}}_{-n,m}(\lambda ) . \end{aligned}$$

(C.30)

If $m\le -1$ and $n\ge 1$, then

$$\begin{aligned} N_{\lambda ,\mu }(q^n t^m) \ne 0 \quad \Longleftrightarrow \quad \lambda \supset {\mathsf {B}}_{n-1,-m-1}(\mu ). \end{aligned}$$

(C.31)

Note that in particular, $N_{\emptyset ,\mu }(t)\ne 0$ if and only if $\mu =(m)$ for some $m \in {\mathbb {Z}}_{\ge 0}$.

1.3 C.3 Duality of functions $p_n(x;s|q,t)$ and $f_n(x;s|q,t)$

Lemma C.2

Let $\varvec{\lambda }$ and $\varvec{\mu }$ satisfy $\ell (\lambda ^{(i)}), \ell (\mu ^{(i)})\le n_i$.

$$\begin{aligned} \left[ x^{-\varvec{\lambda }}x^{\varvec{\mu }} f_{|\varvec{n}|}(x;s(\varvec{\lambda })|q,q/t)p_{|\varvec{n}|}(x;s(\varvec{\mu })|q,q/t) \right] _{x, 1}=\delta _{\varvec{\lambda }, \varvec{\mu }}. \end{aligned}$$

(C.32)

Here, $s(\varvec{\lambda })=(s_j(\varvec{\lambda }))_{1\le j\le |\varvec{n}|}$, $s_{[i,k]_{\varvec{n}}}(\varvec{\lambda })=q^{\lambda ^{(i)}_{k}}t^{1-k}u_i$.

Proof

We denote the LHS by $F(\varvec{\lambda }|\varvec{\mu })$. Inserting the Macdonald operator $D_n^1(s;q,q/t)$ and integrating by parts give

$$\begin{aligned} \begin{aligned}&\left[ x^{-\varvec{\lambda }} f_{|\varvec{n}|}(x;s(\varvec{\lambda })|q,q/t) \left( D_{|\varvec{n}|}^1({\bar{s}};q,q/t) x^{\varvec{\mu }} p_{|\varvec{n}|}(x;s(\varvec{\mu })|q,q/t)\right) \right] _{x, 1} \\&=\left[ \left( \widetilde{D^1_{|\varvec{n}|}}({\bar{s}};q,q/t) x^{-\varvec{\lambda }} f_{|\varvec{n}|}(x;s(\varvec{\lambda })|q,q/t) \right) x^{\varvec{\mu }} p_{|\varvec{n}|}(x;s(\varvec{\mu })|q,q/t) \right] _{x, 1}, \end{aligned} \end{aligned}$$

(C.33)

where ${\bar{s}}_{[i, k]_{\varvec{n}}} = u_i t^{1-k}$. The LHS gives $\epsilon _{\varvec{\mu }}F(\varvec{\lambda }|\varvec{\mu })$, and the RHS gives $\epsilon _{\varvec{\lambda }} F(\varvec{\lambda }|\varvec{\mu })$. Thus we can show $F(\varvec{\lambda }|\varvec{\mu }) = C(\varvec{\lambda }) \delta _{\varvec{\lambda }, \varvec{\mu }}$ with the coefficient $C(\varvec{\lambda }) = F(\varvec{\lambda }|\varvec{\lambda })$. We can show $C(\varvec{\lambda }) = 1$ because both $f_{|\varvec{n}|}(x;s(\varvec{\lambda })|q,q/t)$ and $p_{|\varvec{n}|}(x;s(\varvec{\mu })|q,q/t)$ are in ${\mathbb {C}}(s_1,\ldots , s_{|\varvec{n}|})[[x_2/x_1, x_3/x_2,\ldots , x_{|\varvec{n}|}/x_{|\varvec{n}|-1}]]$ and so is their product. $\quad \square$

1.4 C.4 Some formulas to prove (4.44)

The coincidence between (4.42) and (4.44) can be identified with the following equation.

Proposition C.3

$$\begin{aligned}&\frac{{\mathcal {R}}^{\varvec{n}}_{\varvec{\lambda }}}{{\mathcal {R}}^{\varvec{m}}_{\varvec{\mu }}} {\mathsf {N}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }]}(s_1,\ldots ,s_{|\varvec{n}|+|\varvec{m}|}) \prod _{1\le i<j\le |\varvec{n}|}\frac{(q s_j/ts_i)_{[\varvec{\lambda }]_i-[\varvec{\lambda }]_j}}{(q s_j/s_i)_{[\varvec{\lambda }]_i-[\varvec{\lambda }]_j}}\nonumber \\&= \prod _{i=1}^N t^{-(|\varvec{n}|+n_i)|\lambda ^{(i)}|} (-\gamma ^{-2})^{|\mu ^{(i)}|}(t^{n_i})^{-(N-i)|\mu ^{(i)}|} \nonumber \\&\quad \times t^{(|\lambda ^{(i)}|-|\mu ^{(i)}|)\sum _{s=1}^in_s} \gamma ^{-(N-1)(|\lambda ^{(i)}|+|\mu ^{(i)}|)}t^{2n(\lambda ^{(i)})+ |\lambda ^{(i)}|}q^{n(\mu ^{(i)'})}\nonumber \\&\quad \times \prod _{i=1}^N \left( \frac{f_{\mu ^{(i)}}}{f_{\lambda ^{(i)}}}\right) ^{N-i} \prod _{1\le i<j\le N}(v_i/v_j)^{-|\lambda ^{(i)}|+|\mu ^{(i)}|}\nonumber \\&\quad \times \prod _{i=1}^N\frac{1}{c_{\lambda ^{(i)}}c'_{\mu ^{(i)}}}\frac{\prod _{ i, j=1 }^N N_{\lambda ^{(i)}, \mu ^{(j)}}(t^{n_j}v_i/v_j)}{\prod _{1\le i< j\le N}N_{\lambda ^{(j)}\lambda ^{(i)}}(q v_j/t v_i)\prod _{1\le i< j\le N}N_{\mu ^{(i)}\mu ^{(j)}}(qt^{-n_i+n_j}v_i/t v_j)}. \end{aligned}$$

(C.34)

The following lemmas are formulas for proving (C.34).

Lemma C.4

With $n = \ell (\lambda )$ and $m = \ell (\mu )$, the following holds:

$$\begin{aligned}&\frac{N_{\lambda \mu }(t^{n})}{c_{\lambda }c'_{\mu }} = (-t/q)^{|\mu |}t^{n(\mu )-2n(\lambda )-|\lambda |}(t^{n})^{|\lambda |+|\mu |} q^{-n(\mu ')}\nonumber \\&\quad \times \prod _{1\le i<j\le n}\frac{(q s_j/t s_i)_{\lambda _i-\lambda _j}}{(q s_j/ s_i)_{\lambda _i-\lambda _j}}\prod _{i=1}^{m}\frac{(q/t)_{\mu _{i}}}{(q)_{\mu _i}}\prod _{j=1}^{m}\prod _{i=1}^{n+j-1} \frac{(q s_{j+n}/t s_i)_{\mu _j}}{(q s_{j+n}/s_i)_{\mu _j}}\prod _{1\le i<j\le m}\frac{(q\sigma _i/t\sigma _j)_{\mu _j}}{(q\sigma _i/\sigma _j)_{\mu _j}}, \end{aligned}$$

(C.35)

where

$$\begin{aligned} s_{i} = q^{\lambda _i}t^{1-i}\,\, (i=1,\dots ,n),\quad s_{n+i} = t^{1-n-i}\,\, (i=1,\dots ,m),\quad \text {and}\quad \sigma _i = q^{\mu _i}t^{1-i}.\end{aligned}$$

The equality that we obtain by removing the factors of this type from the both sides of (C.34), can be shown by using following relations.

Lemma C.5

Under the $\lambda _i \rightarrow \lambda _i+1$ or $\mu _i \rightarrow \mu _i +1$, we also have the following induction relations,

$$\begin{aligned}&\frac{N_{\lambda +1_i, \mu }(u)}{N_{\lambda , \mu }(u)} =(1-ut^{\ell (\mu )}\chi _x)\prod _{j=1}^{\ell (\mu )} \frac{1-u q^{-\mu _j}t^{j-1}\chi _x}{1-u q^{-\mu _j}t^j\chi _x}, \end{aligned}$$

(C.36)

$$\begin{aligned}&\frac{N_{\lambda , \mu +1_i}(u)}{N_{\lambda , \mu }(u)} = (1- t^{1-\ell (\lambda )} u/q \chi _y)\prod _{i=1}^{\ell (\lambda )}\frac{1-u q^{\lambda _i-1}t^{2-i}/ \chi _y}{1-u q^{\lambda _i-1}t^{1-i}/ \chi _y}, \end{aligned}$$

(C.37)

where

$$\begin{aligned} \chi _x = q^{\lambda _i}t^{1-i},\quad \chi _y = q^{\mu _i}t^{1-i}. \end{aligned}$$

(C.38)

Notation C.6

In what follows in this proof, we set

$$\begin{aligned}&s_{[i, k]_{\varvec{n}}} =q^{\lambda ^{(i)}_k} t^{1-k}v_i \quad (1 \le k \le n_i, \, i=1,\ldots , N), \end{aligned}$$

(C.39)

$$\begin{aligned}&s_{|\varvec{n}| + [i, k]_{\varvec{m}}} = t^{1- n_{i} -k}v_i \quad (1 \le k \le m_i, \, i=1,\ldots , N),\end{aligned}$$

(C.40)

$$\begin{aligned}&\sigma _{[i, k]_{\varvec{m}}} = q^{\mu ^{(i)}_k} t^{1-k}v_i \quad (1 \le k \le m_i, \, i=1,\ldots , N). \end{aligned}$$

(C.41)

Definition C.7

We set

$$\begin{aligned}&\hat{{\mathsf {N}}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }]}(\varvec{s}) := \prod _{i=1}^N t^{(2\sum _{j=1}^{i-1}n_j+n_i - |\varvec{n}|)|\lambda ^{(i)}|} \nonumber \\&\qquad \times \prod _{l=1}^{N} \prod _{k=1}^{m_l} \left( \prod _{j=1, j\ne l}^{N}\prod _{i=1}^{n_j} \frac{(qs_{[l, k]_{\varvec{m}}}/ts_{[j, i]_{\varvec{n}}};q)_{\mu _k}}{(qs_{[l, k]_{\varvec{m}}}/s_{[j, i]_{\varvec{n}}};q)_{\mu _k}}\prod _{j=1}^{l-1}\prod _{i=1}^{m_j}\frac{(qs_{[l, k]_{\varvec{m}}}/ts_{[j, i]_{\varvec{m}}};q)_{\mu _k}}{(qs_{[l, k]_{\varvec{m}}}/s_{[j, i]_{\varvec{m}}};q)_{\mu _k}} \right) \nonumber \\&\qquad \times \prod _{1\le k<l\le N} \prod _{\begin{array}{c} 1 \le i \le m_k \\ 1 \le j \le m_l \end{array}}\frac{(qt^{-n_k+n_l}\sigma _{[k, i]_{\varvec{m}}}/t\sigma _{[l, j]_{\varvec{m}}})_{\mu ^{(l)}_j}}{(qt^{-n_k+n_l}\sigma _{[k, i]_{\varvec{m}}}/\sigma _{[l, j]_{\varvec{m}}})_{\mu ^{(l)}_j}}\nonumber \\&\quad \times \prod _{1\le k<l\le N} \prod _{\begin{array}{c} 1 \le i \le n_k 1 \le j \le n_l \end{array}} \frac{(qs_{[l, j]_{\varvec{n}}}/t s_{[k, i]_{\varvec{n}}};q)_{-\lambda ^{(l)}_j+\lambda ^{(k)}_i}}{(qs_{[l, j]_{\varvec{n}}}/s_{[k, i]_{\varvec{n}}};q)_{-\lambda ^{(l)}_j+\lambda ^{(k)}_i}}. \end{aligned}$$

(C.42)

Lemma C.8

Under $\lambda ^{(i)}_k \rightarrow \lambda ^{(i)}_k +1$, that is, $s_{[i, k]_{\varvec{n}}}\rightarrow qs_{[i, k]_{\varvec{n}}}$, we have

$$\begin{aligned} \frac{ \gamma ^{-\sum _{j=1}^N (j-1)(|\lambda ^{(j)}|+\delta _{j,i})}{\mathcal {R}}^{\varvec{n}}_{\varvec{\lambda }+1^{(i)}_k}(\varvec{v})}{ \gamma ^{-\sum _{j=1}^N (j-1)|\lambda ^{(j)}|}{\mathcal {R}}^{\varvec{n}}_{\varvec{\lambda }}(\varvec{v})} = \prod _{j=1}^{i-1}\frac{1-q^{-\lambda ^{(i)}_k} t^{-n_j+k-1}/v_{ij}}{1-q^{-\lambda ^{(i)}_k-1}t^{-n_j + k}/v_{ij}}, \end{aligned}$$

(C.43)

with $v_{ij} = v_i/v_j$, and

$$\begin{aligned} \frac{\hat{{\mathsf {N}}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }]}(\ldots ,qs_{[i, k]_{\varvec{n}}},\ldots )}{\hat{{\mathsf {N}}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }]}(\varvec{s})}&=\prod _{l=1}^{i-1}\frac{1-t^{-n_l-m_l}v_l/s_{[i, k]_{\varvec{n}}}}{1-t^{-n_l} v_l/s_{[i, k]_{\varvec{n}}}} \prod _{l=i+1}^N \frac{1-t^{-n_l-m_l}v_l/s_{[i, k]_{\varvec{n}}}}{1-t^{-n_l} v_l/s_{[i, k]_{\varvec{n}}}}\nonumber \\&\quad \times \prod _{l=1, l\ne i}^N \prod _{j=1}^{m_l}\frac{1-q^{\mu _j} s_{[l, j]_{\varvec{m}}}/s_{[i, k]_{\varvec{n}}}}{1-q^{\mu _j} s_{[l, j]_{\varvec{m}}}/ t s_{[i, k]_{\varvec{n}}}}\nonumber \\&\quad \times \left( \prod _{j=1}^{i-1}\prod _{1\le l\le n_j}\frac{1-s_{[j, l]_{\varvec{n}}}/qs_{[i, k]_{\varvec{n}}}}{1-ts_{[j, l]_{\varvec{n}}}/qs_{[i, k]_{\varvec{n}}}} \right) \nonumber \\&\quad \times \left( \prod _{j=i+1}^{N}\prod _{1\le l\le n_j}\frac{1-ts_{[i, k]_{\varvec{n}}}/s_{[j, l]_{\varvec{n}}}}{1-s_{[i, k]_{\varvec{n}}}/s_{[j, l]_{\varvec{n}}}}\right) . \end{aligned}$$

(C.44)

Lemma C.9

Similarly, under $\mu ^{(i)}_k \rightarrow \mu ^{(i)}_k +1$, that is, $\sigma _{[i, k]_{\varvec{n}}}\rightarrow q\sigma _{[i, k]_{\varvec{n}}}$, we have

$$\begin{aligned} \frac{\hat{{\mathsf {N}}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }+1^{(i)}_k]}(\varvec{s})}{\hat{{\mathsf {N}}}^{|\varvec{n}|,|\varvec{m}|}_{[\varvec{\mu }]}(\varvec{s})}&=\prod _{j=1, j\ne i}^{N}\prod _{l=1}^{n_j} \frac{1-q^{\mu ^{(i)}_k+1}s_{[i, k]_{\varvec{m}}}/ts_{[j, l]_{\varvec{n}}}}{1-q^{\mu ^{(i)}_k+1}s_{[i, k]_{\varvec{m}}}/s_{[j, l]_{\varvec{n}}}}\nonumber \\&\quad \times \prod _{j=1}^{i-1}\left( \frac{1-q^{\mu ^{(i)}_k+1}t^{-k}t^{-n_i+n_j}v_{ij}}{1-q^{\mu ^{(i)}_k+1}t^{m_j-k}t^{-n_i+n_j}v_{ij}} \prod _{l=1}^{m_j}\frac{1-t^{-n_j+n_i}\sigma _{[j, l]_{\varvec{m}}}/t \sigma _{[i, k]_{\varvec{m}}}}{1-t^{-n_j+n_i}\sigma _{[j, l]_{\varvec{m}}}/ \sigma _{[i, k]_{\varvec{m}}}}\right) \nonumber \\&\quad \times \prod _{j=i+1}^{N}\left( \frac{1-q t^{-n_i+n_j-1}v_{ij}\chi _y}{1-q t^{-n_i+n_j}t^{m_j-1}v_{ij}\chi _y}\prod _{l=1}^{m_j}\frac{1-q t^{-n_i +n_j}\sigma _{[i, k]_{\varvec{m}}}/\sigma _{[j, l]_{\varvec{m}}}}{1-q t^{-n_i +n_j}\sigma _{[i, k]_{\varvec{m}}}/t\sigma _{[j, l]_{\varvec{m}}}} \right) . \end{aligned}$$

(C.45)

Combining these identities, we complete the proof of (C.34).

Appendix D. Some Proofs of Lemmas and Propositions

1.1 D.1 Proof of Proposition 3.12

By the operator product formulas (C.1)–(C.6), the operator $\Lambda ^{(j)}(z)$ with $j\ne i,i+1$ does not contribute in the commutation relation, and it can be shown that

$$\begin{aligned} \left[ :\Lambda ^{(i)}(z)\Lambda ^{(i+1)}(\gamma ^{-2}z):, S^{(i)}(w)\right] =0. \end{aligned}$$

(D.1)

Hence, it is enough to consider the relation only with $u_i\Lambda ^{(i)}(z)+u_{i+1}\Lambda ^{(i+1)}(z)$.

We have

$$\begin{aligned} \Lambda ^{(i)}(z)S^{(i)}(w)-S^{(i)}(w)\Lambda ^{(i)}(z)t=(1-t)\delta \left( \frac{tw}{qz}\right) :\Lambda ^{(i)}(tw/q)S^{(i)}(w):, \end{aligned}$$

(D.2)

and

$$\begin{aligned} \Lambda ^{(i+1)}(z)S^{(i)}(w)-S^{(i)}(w)\Lambda ^{(i+1)}(z)t^{-1}&=(1-t^{-1})\delta \left( \frac{tw}{z}\right) :\Lambda ^{(i+1)}(tw)S^{(i)}(w):\nonumber \\&=(1-t^{-1})\delta \left( \frac{tw}{z}\right) :\Lambda ^{(i)}(tw)S^{(i)}(qw):. \end{aligned}$$

(D.3)

Therefore, by the property $g_i(qz)=\frac{u_{i+1}}{tu_i}g_i(z)$ with respect to the q-difference, we obtain

$$\begin{aligned}&\left( u_i\Lambda ^{(i)}(z)+u_{i+1}\Lambda ^{(i+1)}(z)\right) S^{(i)}(w)g_i(w)\nonumber \\&\quad -S^{(i)}(w)\left( tu_i\Lambda ^{(i)}(z)+t^{-1}u_{i+1}\Lambda ^{(i+1)}(z)\right) g_i(w)\nonumber \\&=(t-1)u_i(T_{q,w}-1) \delta \left( \frac{tw}{qz} \right) :\Lambda ^{(i)}(tw/q)S^{(i)}(w):g(w). \end{aligned}$$

(D.4)

$\square$

1.2 D.2 Proof of Lemma 3.19

First, we show the relation for $k=0$. In this proof, we write

$$\begin{aligned} \Lambda ^{(i_1,\ldots , i_r)}(z)= :\Lambda ^{(i_1)}(z) \cdots \Lambda ^{(i_r)}((q/t)^{r-1}z): \end{aligned}$$

(D.5)

for $i_1<\cdots < i_r$. By the operator products (C.16)–(C.19) and the relation $:\Phi ^{(0)}(w)\Lambda ^{(1)}(tw):=\Phi ^{(0)}(qw)\Psi ^+(w)$, it can be shown that if $i_1=1$,

$$\begin{aligned}&\Lambda ^{(i_1,\ldots , i_r)}(z) \Phi ^{(0)}(x) - t^{-1} \frac{1-(q/t)^rz/tx}{1-z/tx} \Phi ^{(0)}(x) \Lambda ^{(i_1,\ldots , i_r)}(z)\nonumber \\&=(1-t^{-1})\delta (tx/z) :\Lambda ^{(i_2)}((q/t)tx) \cdots \Lambda ^{(i_r)}((q/t)^{r-1}tx)\Phi ^{(0)}(qx)\Psi ^+(x):. \end{aligned}$$

(D.6)

If $i_1\ge 2$,

$$\begin{aligned} \Lambda ^{(i_1,\ldots , i_r)}(z) \Phi ^{(0)}(x) - \frac{1-(q/t)^rz/tx}{1-z/tx} \Phi ^{(0)}(x) \Lambda ^{(i_1,\ldots , i_r)}(z) =0. \end{aligned}$$

(D.7)

Thus, we obtain the relation in the case $k=0$:

$$\begin{aligned}&X^{(r)}(z) \Phi ^{(0)}(x) - \frac{1-(q/t)^rz/tx}{1-z/tx} \Phi ^{(0)}(x)X^{(r)}(z) \nonumber \\&\quad = u_1 (1-t^{-1})\delta (tx/z) Y^{(r)}(x)\Phi ^{(0)}(qx)\Psi ^+(x). \end{aligned}$$

(D.8)

Applying the screening operators to this relation from the right side, we have the case $k\ne 0$. Indeed, $\Psi ^+(x)$ commutes with $S^{(i)}(y)$:

$$\begin{aligned} \Psi ^+(z) S^{(i)}(y) = S^{(i)}(y)\Psi ^+(z). \end{aligned}$$

(D.9)

Noting that we have

$$\begin{aligned} T_{q,x}g(x,y_1,\ldots ,y_k)=\frac{u_{1}}{u_{k+1}}g(x,y_1,\ldots ,y_k), \end{aligned}$$

(D.10)

and by virtue of (D.9) and commutativity of the screening operators, we can establish the relation for general k.

We can show the commutativity of the screening operators as follows. First, it is clear that

$$\begin{aligned}&\Phi ^{(0)}(x)\cdot \left[ X^{(r)}(z), S^{(1)}(y_1)\cdots S^{(k)}(y_k)g(x,y_1,\ldots ,y_k)\right] \nonumber \\&=\sum _{i=1}^{k}\Phi ^{(0)}(x)S^{(1)}(y_1) \cdots \left[ X^{(r)}(z),S^{(i)}(y_i) \right] \cdots S^{(k)}(y_k)g(x,y_1,\ldots ,y_k). \end{aligned}$$

(D.11)

By calculating the commutation relation as in the proof of Proposition 3.12, the RHS of (D.11) consists of terms as

$$\begin{aligned}&(1-T_{q,y_i})\delta \left( \frac{t\gamma ^{2\ell }y_i}{qz} \right) \Phi ^{(0)}(x)S^{(1)}(y_1) \cdots \nonumber \\&\quad \cdots :\Lambda ^{(j_1,\ldots ,j_{\ell }, i,j_{\ell +2} \ldots , j_r)}(t \gamma ^{2\ell } y_i/q) \, S^{(i)}(y_i): \cdots S^{(k)}(y_k)g(x,y_1,\ldots ,y_k). \end{aligned}$$

(D.12)

Note that $j_{\ell +2}\ge i+2$. Let us investigate the positions of poles in $y_i$. Combining the $\theta$-functions containing $y_i$ in $g(x,y_1,\ldots ,y_k)$ and the operator products among screening currents and $\Phi ^{(0)}(x)$, there appears the factor

$$\begin{aligned}&\frac{1}{\theta _{q}(ty_i/y_{i-1})\theta _{q}(ty_{i+1}/y_{i})} \frac{(ty_i/y_{i-1};q)_{\infty }(ty_{i+1}/y_{i};q)_{\infty }}{(y_i/y_{i-1};q)_{\infty }(y_{i+1}/y_{i};q)_{\infty }}\nonumber \\&=\frac{1}{(qy_{i-1}/ty_{i};q)_{\infty }(qy_{i}/ty_{i+1};q)_{\infty }} \frac{1}{(y_{i}/y_{i-1};q)_{\infty }(y_{i+1}/y_{i};q)_{\infty }}, \quad y_0:=x. \end{aligned}$$

(D.13)

From $\Phi ^{(0)}(x)$ and $\Lambda ^{(i)}(ty_1/q)$ in $\Lambda ^{(j_1,\ldots ,j_{\ell }, i,j_{\ell +2} \ldots , j_r)}$, we have

$$\begin{aligned} \frac{1-t^{\delta _{i,1}-1}ty_i/qx}{1-y_i/tx}. \end{aligned}$$

(D.14)

Noticing that for $i\ge 2$, the operator product of $S^{(i-1)}(y_{i-1})$ and $\Lambda ^{(i)}(ty_i/q)$ are

$$\begin{aligned} S^{(i-1)}(y_{i-1})\Lambda ^{(i)}(ty_i/q)= \frac{1-ty_i/qy_{i-1}}{1-y_i/qy_{i-1}}:S^{(i-1)}(y_{i-1})\Lambda ^{(i)}(ty_i/q):, \end{aligned}$$

(D.15)

we have the following set of poles of (D.11) in $y_i$:

$$\begin{aligned}&y_i=tx, \end{aligned}$$

(D.16)

$$\begin{aligned}&y_i= t^{-1}q^{2+n}y_{i-1}, \quad y_i=q^ny_{i+1}, \end{aligned}$$

(D.17)

$$\begin{aligned}&y_i=t\,q^{-1-n}y_{i+1} \quad (i \ge 1,\, n=0,1,2,\dots ), \end{aligned}$$

(D.18)

and

$$\begin{aligned}&y_i=q^{-n+1}y_{i-1} \quad \text {for}{ i\ge 2}, \end{aligned}$$

(D.19)

$$\begin{aligned}&y_i=q^{-n}x \quad \text {for}{ i=1} \quad (n=0,1,2\dots ). \end{aligned}$$

(D.20)

If $r=1$, it gives us the all poles. In general, this list does not exhaust the possible poles. In case $r\ge 2$, from $S^{(j)}(y_j)$ and $\Lambda ^{(j_m)}$ in $\Lambda ^{(j_1,\ldots ,j_{\ell }, i,j_{\ell +2} \ldots , j_r)}$ with $m\ne \ell +1$, we have extra poles. From the operator product formulas for them, we have

$$\begin{aligned} \prod _{m=1}^{\ell } \frac{1-t^{-1}(q/t)^{-\ell +m-1}y_{i}/y_{j_m}}{1-(q/t)^{-\ell +m-1}y_{i}/y_{j_m}} \frac{1-(q/t)^{-\ell +m-2}y_{i}/y_{j_m-1}}{1-q^{-1}(q/t)^{-\ell +m-1}y_{i}/y_{j_m-1}} \nonumber \\ \times \prod _{m=\ell +2}^{r} \frac{1-t(q/t)^{\ell -m+1}y_{j_m}/y_{i}}{1-(q/t)^{\ell -m+1}y_{j_m}/y_{i}} \frac{1-(q/t)^{\ell -m+2}y_{j_m-1}/y_{i}}{1-q(q/t)^{\ell -m+1}y_{j_m-1}/y_{i}}. \end{aligned}$$

(D.21)

In addition, from $\Phi ^{(0)}(x)$ and $\Lambda ^{(j_m)}$, the following factor arises:

$$\begin{aligned} \prod _{m\ne \ell +1}\frac{1-t^{\delta _{j_m,1}-1} (q/t)^{-\ell +m-2} y_i/x}{1-t^{-1} (q/t)^{-\ell +m-1} y_i/x}. \end{aligned}$$

(D.22)

Summarizing these, we can show that the poles in $y_i$ are in the following positions (Though not all following points are poles, all poles should be in the followings or (D.16)–(D.20)). For $i\ge 1$,

$$\begin{aligned}&y_i=(q/t)^{-n-1} y_{j+1}, \quad y_i=q (q/t)^{-n-1}y_j \quad (j>i), \end{aligned}$$

(D.23)

$$\begin{aligned}&y_i=q (q/t)^{-n} x, \quad (n=0,1,2\ldots ), \end{aligned}$$

(D.24)

and for $i\ge 2$,

$$\begin{aligned}&y_i=(q/t)^{n+1} y_j \quad (1\le j<i ), \quad y_i=q (q/t)^{n+1}y_{j-1} \quad (2\le j<i ), \end{aligned}$$

(D.25)

$$\begin{aligned}&y_i=q(q/t)^{n+1}x \quad (n=0,1,2\ldots ). \end{aligned}$$

(D.26)

For the given integration contour, the poles (D.17), (D.25) and (D.26) are in the disk $\{z; |z|<|qy_i| \}$. On the other hand, the poles (D.16), (D.18), (D.19), (D.20), (D.23) and (D.24) are in $\{ z; |z|>|y_i| \}$. Therefore, the change of variable $y_i \rightarrow q y_i$ is not affected by these poles, and the commutation relation (D.11) becomes zero after the integrals. $\quad \square$

1.3 D.3 Proof of Proposition 3.27

By taking the constant terms of with respect to $x_i$, the proportional constant ${\mathcal {R}}^{\varvec{n}}_{\varvec{\lambda }}$ is calculated as the expansion coefficient in front of in the basis of generalized Macdonald functions. We first consider only the operators that contain the creation operators $a^{(N)}_{-n}$’s with respect to the N-th Fock space. That is, we take the constant terms of

(D.27)

Here, we used the expansion formula (3.24), and $\widetilde{\mathrm {M}}=\mathrm {Mat}(n_N,N-1;{\mathbb {Z}})$ is the set of $n_N \times (N-1)$ matrices with integral entries. We denote $x_{[N,i]_{\varvec{n}}}$ by $y_{i,0}$ in this proof. Further we set

$$\begin{aligned}&\alpha ^{(k)}_{i,l}=t^{-n_l+i}\frac{u_l}{u_k}, \end{aligned}$$

(D.28)

$$\begin{aligned}&R_r(\alpha )=\frac{(\alpha ;q)_r}{(q\alpha /t;q)_r}. \end{aligned}$$

(D.29)

Let $C(z)=\sum _{k\ge 0} C_k z^k$, ${\widetilde{C}}(z)=\sum _{k\ge 0} {\widetilde{C}}_kz^k$ and $C^{(\pm )}(z)=\sum _{k\ge 0} C^{(\pm )}_kz^k$ be the formal power series defined by

$$\begin{aligned}&C(z) =\frac{(q z/ t ; q)_\infty }{(t z; q)_\infty }, \quad {\widetilde{C}}(z) = (1- z) \frac{(q z/ t ; q)_\infty }{(t z; q)_\infty }, \end{aligned}$$

(D.30)

$$\begin{aligned}&C^{(+)}(z) = \frac{(t z ; q)_\infty }{(z ; q)_\infty }, \quad C^{(-)}(z) = \frac{(qz ; q)_\infty }{(q z/t ; q)_\infty }. \end{aligned}$$

(D.31)

These series correspond to the operator product formulas among $\Phi ^{(0)}$ and $S^{(i)}$’s. Moreover, we write

$$\begin{aligned}&E^{(m)}_i(e)=-\sum _{s=i+1}^{n_N} e^{(m)}_{i,s}+\sum _{s=1}^{i-1}e_{s,i}^{(m)}, \end{aligned}$$

(D.32)

$$\begin{aligned}&K^{(m)}_i(k,\ell )=-\sum _{s=i+1}^{n_N} \ell ^{(m-1)}_{i,s}+\sum _{s=1}^{i-1}k_{s,i}^{(m)},\end{aligned}$$

(D.33)

$$\begin{aligned}&L^{(m)}_i(k,\ell )=-\sum _{s=i+1}^{n_N} k^{(m)}_{i,s}+\sum _{s=1}^{i-1}\ell _{s,i}^{(m-1)} \end{aligned}$$

(D.34)

for $e=((e^{(m)}_{i,j})_{i,j=1}^{n_N})_{1\le m\le N-1}, k=((k^{(m)}_{i,j})_{i,j=1}^{n_N})_{1\le m\le N-1} \in M_{n_N}^{N-1}$, and $\ell =((\ell ^{(m)}_{i,j})_{i,j=1}^{n_N})_{0\le m\le N-2}\in M_{n_N}^{N-1}$. Here $M_n$ is the set of strictly upper triangular $n\times n$ matrices with nonnegative integral entries. With these notations, (D.27) can be rewritten as

(D.35)

Here, $\widetilde{\mathrm {M}}_{\ge 0}=\mathrm {Mat}(n_N, N-1;{\mathbb {Z}}_{\ge 0})$, $\Phi ^{(0)}(z)=\sum _{n \in {\mathbb {Z}}} \Phi ^{(0)}_{n}z^{-n}$, and $S^{(i)}(z)=\sum _{n \in {\mathbb {Z}}} S^{(i)}_{n}z^{-n}$. Since the integral gives us the constant terms in $y_{i,m}$, we have

$$\begin{aligned} r_{i,N-1}&=-E^{(N-1)}_i-L^{(N-1)}_i-\mathsf {a}^{(N-1)}_i, \end{aligned}$$

(D.36)

$$\begin{aligned} r_{i,N-2}&=-E^{(N-1)}_i-L^{(N-2)}_i-K^{(N-1)}_i-\mathsf {a}^{(N-2)}_i+r_{i,N-1} \nonumber \\&=-E^{(N-1)}_i-E^{(N-2)}_i-L^{(N-1)}_i-L^{(N-2)}_i-K^{(N-1)}_i-\mathsf {a}^{(N-1)}_i-\mathsf {a}^{(N-2)}_i,\end{aligned}$$

(D.37)

$$\begin{aligned}&\cdots \nonumber \\ r_{i,1}&=-\sum _{m=1}^{N-1}\left( E_i^{(m)} +L_i^{(m)} +K_i^{(m)} +\mathsf {a}^{(m)}_i \right) +K_i^{(1)}, \end{aligned}$$

(D.38)

and

$$\begin{aligned} \mathsf {a}^{(0)}_i+E^{(0)}_i+\sum _{m=1}^{N-1}\left( E_i^{(m)} +L_i^{(m)} +K_i^{(m)} +\mathsf {a}^{(m)}_i \right) -\lambda ^{(N)}_i=0. \end{aligned}$$

(D.39)

Since $\sum _{i=1}^{n_N} E^{(m)}_i=0$ and $\sum _{i=1}^{n_N} (L_i^{(m)} +K_i^{(m)}) =0$, it is shown that

$$\begin{aligned} |\lambda ^{(N)}|-\sum _{i=1}^{n_N}\mathsf {a}^{(N-1)}_i=\sum _{m=0}^{N-2}\sum _{i=1}^{n_N}\mathsf {a}^{(m)}_i\ge 0. \end{aligned}$$

(D.40)

Therefore, $\sum _{i=1}^{n_N}\mathsf {a}_i^{(N-1)}$ takes its maximum value $|\lambda ^{(N)}|$ when $\mathsf {a}^{(m)}_i=0$ for all i and $m \le N-2$. Since only the operators $:\prod _{i=1}^{n_N}S^{(N-1)}_{-\mathsf {a}^{(N-1)}_i}:$ have the creation operators acting on the N-th Fock space, it is clear that the maximum degree in the N-th Fock component is $|\lambda ^{(N)}|$.

Before taking expansion coefficients in the basis of generalized Macdonald functions, we investigate the one in the basis of products of ordinary Macdonald functions . Consider the terms of level $|\lambda ^{(N)}|$ with respect to the N-th Fock space. Then, $\mathsf {a}^{(N-1)}_{i}$ satisfies

$$\begin{aligned} \mathsf {a}^{(N-1)}_i=\lambda ^{(N)}_i-E^{(0)}_i-\sum _{m=1}^{N-1}\left( E_i^{(m)} +L_i^{(m)} +K_i^{(m)} \right) . \end{aligned}$$

(D.41)

Furthermore, by the form of $E^{(m)}_i$, $K^{(m)}_i$ and $L^{(m)}_i$, it can be seen that only the following vectors appear:

(D.42)

By the Pieri formula (Fact B.4), we can write the terms of level $|\lambda ^{(N)}|$ with respect to the N-th Fock space as

(D.43)

Here, $g^{(N)}_n$ is defined by

$$\begin{aligned} \sum _{n\ge 0} z^n g^{(N)}_{n}= :\exp \left( \sum _{n>0} \frac{1-t^n}{1-q^n} a^{(N)}_{-n}z^n\right) :. \end{aligned}$$

(D.44)

Therefore, there appears only in the case that $\mu =\lambda ^{(N)}$ on (D.42), i.e., the case that

$$\begin{aligned} e^{(m)}_{i,j}=\ell ^{(m)}_{i,j}=k^{(m)}_{i,j}=0 \end{aligned}$$

(D.45)

for all i, j, m. Then

$$\begin{aligned} r_{i,m}=-\mathsf {a}^{(N-1)}_i=-\lambda ^{(N)}_i \end{aligned}$$

(D.46)

for all i, m.

From the above discussion, we have

(D.47)

Here ${\mathcal {O}}\left( P \right)$ expresses the terms with $\varvec{\mu }$ satisfying the proposition P. By repeating the similar argument $N-1$ times, we obtain

(D.48)

The existence theorem of generalized Macdonald functions (Fact 3.6) shows that the coefficient in front of in the basis of generalized Macdonald functions is the same as the one in front of in (D.48). $\quad \square$

1.4 D.4 Proof of Lemma 4.11

First, it can be shown that

$$\begin{aligned}&\prod _{1\le i<j\le n+m} {(q^{-\theta _{j}}q s_j/t s_i;q)_{\theta _{i}} \over (q^{-\theta _{j}}s_j/s_i;q)_{\theta _{i}} } \nonumber \\&=\prod _{1\le i<j\le n+m} \frac{q^{\theta _i}s_i^{-1}-q^{\theta _j}s_j^{-1}}{s_i^{-1}-s_j^{-1}} \frac{(ts_i/s_j;q)_{\theta _j}}{(qs_i/s_j;q)_{\theta _j}} t^{-\theta _j} \frac{(qs_j/ts_i;q)_{\theta _i-\theta _j}}{(qs_j/s_i;q)_{\theta _i-\theta _j}} \end{aligned}$$

(D.49)

and

$$\begin{aligned} \prod _{1\le i<j\le n} \frac{(qq^{-\theta _{j}}s_{j}/tq^{-\theta _{i}}s_{i};q)_{\infty }}{(qq^{-\theta _{j}}s_{j}/q^{-\theta _{i}}s_{i};q)_{\infty }} =\prod _{1\le i<j\le n} \frac{(q s_j/t s_i;q)_{\infty }}{(q s_j/ s_i;q)_{\infty }} \frac{(qs_j/s_i;q)_{\theta _i-\theta _j}}{(qs_j/ts_i;q)_{\theta _i-\theta _j}}. \end{aligned}$$

(D.50)

By (D.49) and (D.50), we have

(D.51)

Furthermore, we can see

$$\begin{aligned}&\prod _{k=1}^{m-1} \prod _{i=1}^{n+k} \frac{(qq^{-\theta _{n+k}}s_{n+k}/tq^{-\theta _{i}}s_{i};q)_{\sigma _k}}{(qq^{-\theta _{n+k}}s_{n+k}/q^{-\theta _{i}}s_{i};q)_{\sigma _k}} \nonumber \\&=\lim _{h\rightarrow 1} \prod _{k=1}^{m-1} \prod _{i=1}^{n+k} \frac{( q q^{\rho _k} s_{n+k}/ts_i ;q)_{\theta _i}}{(h q q^{\rho _k} s_{n+k}/s_i ;q)_{\theta _i}} \frac{( q s_{n+k}/ts_i ;q)_{\rho _k}}{(h q s_{n+k}/s_i ;q)_{\rho _k}} \frac{(qs_{n+k}/s_i;q)_{\theta _i-\theta _{n+k}}}{(qs_{n+k}/ts_i;q)_{\theta _i-\theta _{n+k}}} \end{aligned}$$

(D.52)

and

$$\begin{aligned}&\prod _{1\le k<l \le m-1} \frac{(t q^{-\sigma _k}q^{-\theta _{n+l}}s_{n+l}/q^{-\theta _{n+k}}s_{n+k};q)_{\sigma _l}}{(q^{-\sigma _k}q^{-\theta _{n+l}}s_{n+l}/q^{-\theta _{n+k}}s_{n+k};q)_{\sigma _l}}\nonumber \\&= \prod _{1\le k <l \le m-1} \frac{(t q^{-\rho _k} s_{n+l}/ s_{n+k};q)_{\rho _l}}{(q^{-\rho _k} s_{n+l}/ s_{n+k};q)_{\rho _l}} \frac{(q q^{\rho _k} s_{n+k}/ ts_{n+l};q)_{\theta _{n+l}}}{(q^{\rho _k} s_{n+k}/ s_{n+l};q)_{\theta _{n+l}}} \times t^{\theta _{n+l}}. \end{aligned}$$

(D.53)

Combining (D.51), (D.52) and (D.53) yields Lemma 4.11. $\quad \square$

1.5 D.5 Proof of Lemma 4.13

Set for short

$$\begin{aligned}&A=\prod _{k=1}^{m-1}\frac{q^{\nu _k+\rho _k}s_{n+k}-q^{\nu _m} s_{n+m}}{q^{\rho _{k}}s_{n+k}-s_{n+k}} \nonumber \\&\qquad \times \prod _{k=1}^{m-1} \frac{(t;q)_{\nu _{k}}}{(q;q)_{\nu _{k}}} \frac{(q q^{\rho _k}s_{n+k}/ts_{n+m};q)_{\nu _{k}}}{(q q^{\rho _k}s_{n+k}/s_{n+m};q)_{\nu _{k}}} \frac{(t q^{-\rho _k}s_{n+m}/s_{n+k};q)_{\nu _{m}}}{(q q^{-\rho _k}s_{n+m}/s_{n+k};q)_{\nu _{m}}}, \end{aligned}$$

(D.54)

$$\begin{aligned}&B= \prod _{1\le k<l \le m-1} \frac{q^{\nu _k+\rho _k}s_{n+k}-q^{\nu _l+\rho _l} s_{n+l}}{q^{\rho _{k}}s_{n+k}-q^{\rho _l}s_{n+l}}\cdot \prod _{\begin{array}{c} k\ne l \end{array}} \frac{(t q^{\rho _k-\rho _l}s_{n+k}/s_{n+l};q)_{\nu _{k}}}{(q q^{\rho _k-\rho _l}s_{n+k}/s_{n+l};q)_{\nu _{k}}}, \end{aligned}$$

(D.55)

$$\begin{aligned}&C=\prod _{k=1}^{m-1}\prod _{i=1}^{n+m-1} \frac{(hq q^{\rho _k}s_{n+k}/ts_{i};q)_{\nu _{k}}}{(h q q^{\rho _k}s_{n+k}/s_{i};q)_{\nu _{k}}}, \end{aligned}$$

(D.56)

$$\begin{aligned}&D=\frac{(q/t;q)_{\nu _m}}{(q;q)_{\nu _m}} \prod _{i=1}^{n+m-1} \frac{(h q s_{n+m}/t s_{i};q)_{\nu _{m}}}{(h q s_{n+m}/s_{i};q)_{\nu _{m}}}. \end{aligned}$$

(D.57)

Then $\phi ^{m,n+m-1}_{\nu }=ABCD$. First, we can get

$$\begin{aligned} A=&\prod _{k=1}^{m-1} (q/t)^{\theta _k} \frac{(t;q)_{\theta _{k}}}{(q;q)_{\theta _{k}}} \frac{(t q^{-\mu _k+\mu _m}s_{n+m}/s_{n+k};q)_{\theta _{k}}}{(q q^{-\mu _k+\mu _m}s_{n+m}/s_{n+k};q)_{\theta _{k}}} \nonumber \\&\times \prod _{k=1}^{m-1} \frac{(t q^{-\mu _k}s_{n+m}/s_{n+k};q)_{\mu _{m}}}{(q^{-\mu _k}s_{n+m}/s_{n+k};q)_{\mu _{m}}}. \end{aligned}$$

(D.58)

The first product in (D.58) reproduces the factors in $d_m((\theta _i); (q^{\mu _i}s_{n+i})|q,t)$, i.e., the first product in (3.36). Next, we have

$$\begin{aligned}&\lim _{h\rightarrow 1} \widetilde{{\mathsf {N}}}^{n,m-1}_{\rho }(h; s_1,\ldots , s_{n+m-1}) C = {\mathsf {N}}^{n,m-1}_{\mu }(s_1,\ldots , s_{n+m-1}) E , \end{aligned}$$

(D.59)

where

$$\begin{aligned}&E\equiv \prod _{1\le k<l\le m-1} t^{-\theta _k} \frac{(t q^{-\mu _k+\mu _l}s_{n+l}/s_{n+k};q)_{\theta _k-\theta _l}}{( q^{-\mu _k+\mu _l}s_{n+l}/s_{n+k};q)_{\theta _k-\theta _l}}. \end{aligned}$$

(D.60)

BE reproduces the factors in $d_m((\theta _i); (q^{\mu _i}s_{n+i})|q,t)$:

$$\begin{aligned} BE=\prod _{1\le i <j \le m} \frac{(tq^{-\mu _k+\mu _l}s_{n+l}/s_{n+k};q)_{\theta _k}}{(qq^{-\mu _k+\mu _l}s_{n+l}/s_{n+k};q)_{\theta _k}} \frac{(qq^{-\mu _k+\mu _l-\theta _l}s_{n+l}/ts_{n+k};q)_{\theta _k}}{(q^{-\mu _k+\mu _l-\theta _l}s_{n+l}/s_{n+k};q)_{\theta _k}}. \end{aligned}$$

(D.61)

This corresponds to the second product in (3.36). The product of ${\mathsf {N}}^{n,m-1}_{\mu }$, D and the remaining factor in (D.58) is

$$\begin{aligned}&{\mathsf {N}}^{n,m-1}_{\mu }(s_1,\ldots , s_{n+m-1}) \cdot \prod _{k=1}^{m-1}\frac{(t q^{-\mu _k}s_{n+m}/s_{n+k};q)_{\mu _{m}}}{(q^{-\mu _k}s_{n+m}/s_{n+k};q)_{\mu _{m}}}\cdot \lim _{h\rightarrow 1}D \nonumber \\&= {\mathsf {N}}^{n,m}_{\mu }(s_1,\ldots , s_{n+m}). \end{aligned}$$

(D.62)

Therefore, Lemma 4.13 follows. $\quad \square$

Appendix E. Kac Determinant Revisited

The formula for the Kac determinant with respect to the vectors has been discussed in [O]. That shows the fact that form a basis on the Fock space (Fact 2.10). For the sake of reader’s convenience, we revisit the proof, clarifying the choice of the integral cycles. Here, we construct the q-invariance cycles by using the elliptic theta function.

Definition E.1

Let $1\le k\le N-1$ and $u_k=q^s t^{-r} u_{k+1}$ ($r, s \in {\mathbb {Z}}_{>0}$ ). Define the vector by the integral

(E.1)

Here and hereafter, we use the shorthand notation

$$\begin{aligned} \oint \frac{dz}{z}:=\oint _{T} \prod _{i=1}^r \frac{dz_i}{2 \pi \sqrt{-1} z_i}, \end{aligned}$$

(E.2)

where the cycle is the r-dimensional torus $T: |z_1|=\cdots =|z_r|=1$. Note that $\varvec{u}$ is the spectral parameter of the codomain of $S^{(k)}(z_1)$.

Proposition E.2

The vector does not vanish. In particular, this is of level rs.

Let us prepare a lemma with respect to the symmetrization of theta functions. Set

$$\begin{aligned} {\widehat{F}}_{r,s} (z_1,\ldots , z_r)&:=\frac{1}{r!}\sum _{\sigma \in \mathfrak {S}_r} \prod _{i=1}^r \theta _q(q^s t^{2i-r} z_{\sigma (i)}) \cdot \prod _{\begin{array}{c} i<j \\ \sigma (i) > \sigma (j) \end{array}} t^{-1}\frac{\theta _q(t z_{\sigma (i)}/z_{\sigma (j)})}{ \theta _q(t^{-1}z_{\sigma (i)}/z_{\sigma (j)})}. \end{aligned}$$

(E.3)

Lemma E.3

$$\begin{aligned} {\widehat{F}}_{r,s} (z_1,\ldots , z_r)= \frac{1}{r !} \prod _{i=2}^{r} \frac{\theta _q (t^i)}{\theta _q(t)} \cdot \prod _{1\le i<j \le r} \frac{\theta _q (z_i/z_j)}{\theta _q (tz_i/z_j)} \cdot \prod _{i=1}^r \theta _q(q^s t z_i). \end{aligned}$$

(E.4)

As for the proof of this lemma, see the proof of Lemma 4 in [JLMP].

Proof of Proposition E.2

First, by using the operator products of screening currents and Lemma E.3, we have

(E.5)

Here, $\Delta (z)$ is defined in (B.15). Note that can be regarded as the kernel function for the Macdonald functions. Hence, it is expanded in terms of the Macdonald functions (See Fact B.3). Note that $\prod _{i=1}^rz_i^{-s}$ is the Macdonald polynomial with a rectangular Young diagram in r variables. Therefore, (E.5) can be written as

(E.6)

where $P_{\lambda }^{(r)}(z)$ and $Q^{(r)}_{\lambda }(z)$ denote the Macdonald polynomials in r variables,

$$\begin{aligned} \alpha _{-n}^{(k)}:=\gamma ^{kn}(-\gamma ^n a^{(k)}_{-n}+a^{(k+1)}_{-n}), \end{aligned}$$

(E.7)

and $\langle -,- \rangle '_r$ denotes the scalar product defined in Appendix B. Since the Macdonald polynomials are pairwise orthogonal for $\langle -,- \rangle '_r$ and the inner product $\langle P^{(r)}_{(s^r)},Q^{(r)}_{(s^r)} \rangle '_r$ can be evaluated (Fact B.6) to be nonvanishing. Thus, and of level rs. $\quad \square$

We show the following commutativity with the algebra $\mathsf {U}(N)$. The proof is similar to the case corresponding to the Minimal model, given in [JLMP].

Proposition E.4

Let $r,s \in {\mathbb {Z}}_{>0}$ and $k,j \in \{1,\ldots , r\}$. Further we assume $|t|<|q|$. Then

$$\begin{aligned} \Big [ X^{(j)}(z), \oint \frac{dw}{w} S^{(k)}(w_1) \cdots S^{(k)}(w_r) \prod _{i=1}^{r} \frac{\theta _q( t^{2i}\frac{u_k}{u_{k+1}}w_i)}{\theta _q(tw_i)} \Big ] =0, \end{aligned}$$

(E.8)

where the spectral parameter of the codomain of $S^{(k)}(w_1)$ is $\varvec{u}$ with $u_k=q^st^{-r}u_{k+1}$.

Proof

As in the proof of Proposition 3.12, it suffices to consider the relation only with $\Lambda ^{(k)}(z)+\Lambda ^{(k+1)}(z)$. By (D.4), we have

$$\begin{aligned}&\Big [ \Lambda ^{(k)}(z)+\Lambda ^{(k+1)}(z), \oint \frac{dw}{w} S^{(k)}(w_1) \cdots S^{(k)}(w_r) \cdot \prod _{i=1}^{r}\frac{\theta _q( t^{2i}\frac{u_k}{u_{k+1}}w_i)}{\theta _q(tw_i)}\Big ] \nonumber \\ {}&=\sum _{m=1}^r\oint \frac{dw}{w} (t-1)t^{m-1} u_k \left( T_{q,w_m}-1 \right) \delta \left( \frac{tw_m}{qz} \right) \Delta (w)\nonumber \\ {}&\quad \times \prod _{1\le i <j\le r} \frac{\theta _q( tw_i/w_j)}{\theta _q(w_i/w_j)} \cdot \prod _{i=1}^r\frac{\theta _q( q^st^{2i-r}w_i)}{\theta _q(tw_i)} \times \prod _{i=1}^{m-1}\frac{1-t^{-1}w_m/w_i}{1-w_m/w_i}\\ {}&\quad \cdot \prod _{i=m+1}^{r}\frac{1-tw_i/w_m}{1-w_i/w_m} \cdot :\Lambda ^{(k)}(tw_m/q) \prod _{i=1}^rS^{(k)}(z_i):.\nonumber \end{aligned}$$

(E.9)

By symmetrizing the variables $w_i$’s, we have

$$\begin{aligned}&\frac{1}{r!}\sum _{m=1}^r \sum _{\sigma \in \mathfrak {S}_r} \oint \frac{dw}{w} (t-1)u_k \left( T_{q,w_{\sigma (m)}}-1 \right) \delta \left( \frac{tw_{\sigma (m)}}{qz} \right) \Delta (w) \nonumber \\&\qquad \times \prod _{1\le i<j\le r} \frac{\theta _q( tw_{\sigma (i)}/w_{\sigma (j)})}{\theta _q(w_{\sigma (i)}/w_{\sigma (j)})} \cdot \prod _{i=1}^r\frac{\theta _q( q^st^{2i-r}w_{\sigma (i)})}{\theta _q(tw_{\sigma (i)})} \nonumber \\&\qquad \times \prod _{i\ne m}\frac{1-tw_{\sigma (i)}/w_{\sigma (m)}}{1-w_{\sigma (i)}/w_{\sigma (m)}}\cdot :\Lambda ^{(k)}(tw_{\sigma (m)}/q) \prod _{i=1}^rS^{(k)}(z_i): \nonumber \\&=\frac{1}{r!}\sum _{l=1}^r \sum _{m=1}^r \sum _{\begin{array}{c} \sigma \in \mathfrak {S}_r\\ \sigma (m)=l \end{array}} \oint \frac{dw}{w} (t-1)u_k \left( T_{q,w_l}-1 \right) \delta \left( \frac{tw_{l}}{qz} \right) \Delta (w)\nonumber \\&\qquad \times \prod _{1\le i<j\le r} \frac{\theta _q( tw_{\sigma (i)}/w_{\sigma (j)})}{\theta _q(w_{\sigma (i)}/w_{\sigma (j)})} \cdot \prod _{i=1}^r\frac{\theta _q( q^st^{2i-r}w_{\sigma (i)})}{\theta _q(tw_{\sigma (i)})} \nonumber \\&\qquad \cdot \prod _{\sigma (i)\ne l}\frac{1-tw_{\sigma (i)}/w_{l}}{1-w_{\sigma (i)}/w_{l}} \cdot :\Lambda ^{(k)}(tw_{l}/q) \prod _{i=1}^rS^{(k)}(z_i):\nonumber \\&=\frac{1}{r!}\sum _{l=1}^r \oint \frac{dw}{w} (t-1)u_k \left( T_{q,w_l}-1 \right) \delta \left( \frac{tw_{l}}{qz} \right) \Delta (w) \nonumber \\&\qquad \times \left( \sum _{\sigma \in \mathfrak {S}_r} \prod _{1\le i <j\le r} \frac{\theta _q( tw_{\sigma (i)}/w_{\sigma (j)})}{\theta _q(w_{\sigma (i)}/w_{\sigma (j)})} \cdot \prod _{i=1}^r\frac{\theta _q( q^st^{2i-r}w_{\sigma (i)})}{\theta _q(tw_{\sigma (i)})} \right) \nonumber \\&\qquad \times \prod _{i\ne l}\frac{1-tw_{i}/w_{l}}{1-w_{i}/w_{l}} \cdot :\Lambda ^{(k)}(tw_{l}/q) \prod _{i=1}^rS^{(k)}(z_i):. \end{aligned}$$

(E.10)

By Lemma E.3, this can be rewritten as

$$\begin{aligned}&\frac{1}{r!}\sum _{l=1}^r \oint \frac{dw}{w} (t-1)u_k \left( T_{q,w_l}-1 \right) \delta \left( \frac{tw_{l}}{qz} \right) \Delta (w) \nonumber \\&\qquad \times \prod _{i=2}^r\frac{\theta _q(t^i)}{\theta _q(t)} \cdot \prod _{i=1}^r q^{\frac{s(1-s)}{2}}(tz_i)^{-s} \cdot \prod _{i\ne l}\frac{1-tw_{i}/w_{l}}{1-w_{i}/w_{l}} \cdot :\Lambda ^{(k)}(tw_{l}/q) \prod _{i=1}^rS^{(k)}(z_i):. \end{aligned}$$

(E.11)

In this expression, we have poles in $w_l$ at $w_l=0$, $w_l=q^nt w_i$ and $w_l=q^{-n+1}t^{-1}w_i$ ($i\ne l$, $n=1,2,\ldots$). Since $|t|<|q|$, they do not change the integral while we q-shift the cycle as $w_l\rightarrow q w_l$. Therefore, the integral (E.11) is zero. $\quad \square$

Proposition E.2 and Proposition E.4 show the existence of the singular vectors of the algebra $\mathsf {U}(N)$.

Corollary E.5

The vector is a singular vector of level rs, i.e.,

(E.12)

for all $n >0$ and $i=1,\ldots , N$.

We revisit the proof of the following formula for the Kac determinant .

Proposition E.6

We have

$$\begin{aligned} \mathrm {det}_n&= \prod _{\varvec{\lambda }\vdash n} \prod _{k=1}^N b_{\lambda ^{(k)}}(q) b'_{\lambda ^{(k)}}(t^{-1}) \nonumber \\&\times \prod _{\begin{array}{c} 1\le r,s\\ rs\le n \end{array}} \left( (u_1 u_2 \cdots u_N)^{2} \prod _{1\le i < j \le N} (u_i-q^st^{-r}u_j)(u_i-q^{-r}t^s u_j) \right) ^{P^{(N)}(n-rs)}. \end{aligned}$$

(E.13)

Here $b_{\lambda }(q) := \prod _{i\ge 1} \prod _{k=1}^{m_i} (1-q^k)$, $b'_{\lambda }(q) := \prod _{i\ge 1} \prod _{k=1}^{m_i} (-1+q^k)$. $P^{(N)}(n)$ is the number of N-tuples of Young diagrams of size n, i.e., $\# \big \{ \varvec{\lambda }=(\lambda ^{(1)}, \ldots , \lambda ^{(N)}) \big | |\varvec{\lambda }| =n\big \}$. In particular, if $N=1$,

$$\begin{aligned} \mathrm {det}_n=\prod _{\lambda \vdash n} b_{\lambda }(q) b'_{\lambda }(t^{-1}) \times u_1^{2\sum _{\lambda \vdash n}\ell (\lambda )}. \end{aligned}$$

(E.14)

Proof

The inner product can be calculated by commutation relations of $X^{(i)}_n$. The parameters $u_1, \ldots , u_N$ arise from the eigenvalues of $X^{(i)}_0$. Therefore, it can be seen that is a polynomial in

$$\begin{aligned} m_{(1^i)}(u_1,\ldots ,u_N)=\sum _{j_1<\cdots <j_i} u_{j_1}\cdots u_{j_i} \end{aligned}$$

(E.15)

over ${\mathbb {Q}}(q^{\frac{1}{2}},t^{\frac{1}{2}})$, and thus so does the $\mathrm {det}_n$. Define the action of the symmetric group $\mathfrak {S}_N$ (the Weyl group of type $A_{N-1}$) on polynomials in $u_j$ in the usual way. Since $m_{(1^i)}(u_1,\ldots ,u_N)$ is invariant with respect to this action, $\mathrm {det}_n$ is also invariant, i.e., a symmetric polynomial in $u_j$.

Furthermore, let us introduce the new parameters $u_i'$ and $u''$ by

$$\begin{aligned} \prod _{i=1}^N u_i'=1, \quad u_i=u_i' u''. \end{aligned}$$

(E.16)

Then can be decomposed as

(E.17)

Therefore, $\mathrm {det}_n$ can be written as

$$\begin{aligned} \mathrm {det}_n=(u'')^{2 \sum _{|\varvec{\lambda }|=n}\sum _{k=1}^N k \ell (\lambda ^{(k)})} \times F(u'_1,\ldots , u'_N), \end{aligned}$$

(E.18)

where $F(u'_1,\ldots , u'_N)$ is some polynomial in $u'_i$. Note that the maximum degree of $F(u'_1,\ldots , u'_N)$ with respect to each $u'_i$ is $2\sum _{|\varvec{\lambda }|=n}\sum _{k=1}^N \ell (\lambda ^{(k)})$.

By Corollary E.5, it can be seen that for $r, s \in {\mathbb {Z}}_{>0}$ with $rs\le n$, the Kac determinant $\mathrm {det}_n$ has the factors

$$\begin{aligned} (u_k-q^{s}t^{-r}u_{k+1})^{P^{(N)}(n-rs)}=\left( u''(u'_k-q^{s}t^{-r}u'_{k+1}) \right) ^{P^{(N)}(n-rs)} \end{aligned}$$

(E.19)

in the usual way. By the $\mathfrak {S}_N$ invariance, $\mathrm {det}_n$ has also the factor

$$\begin{aligned} (u_i-q^{\pm s}t^{\mp r}u_{j})^{P^{(N)}(n-rs)}=\left( u''(u'_i-q^{\pm s}t^{\mp r}u'_{j}) \right) ^{P^{(N)}(n-rs)} \end{aligned}$$

(E.20)

for $i\ne j$. Noticing the degree of $F(u'_1,\ldots , u'_N)$, we can see that

$$\begin{aligned}&\mathrm {det}_n = g_{N,n}(q,t) \times (u'')^{2\sum _{\varvec{\lambda }\vdash n} \sum _{i=1}^N i\, \ell (\lambda ^{(i)})} \nonumber \\&\qquad \quad \times \prod _{1 \le i<j\le N} \prod _{\begin{array}{c} 1\le r,s \\ rs \le n \end{array}} \left( (u'_{i}-q^{s} t^{-r}u'_{j})(u'_{i}-q^{-r} t^{s}u'_{j}) \right) ^{P^{(N)}(n-rs)} \nonumber \\&= g_{N,n}(q,t) \times \prod _{\begin{array}{c} 1\le r,s \\ rs \le n \end{array}} \left( (u_1u_2\cdots u_N)^2 \prod _{1 \le i<j\le N} (u_{i}-q^{s} t^{-r}u_{j})(u_{i}-q^{-r} t^{s}u_{j}) \right) ^{P^{(N)}(n-rs)}. \end{aligned}$$

(E.21)

Here, $g_{N,n}(q,t) \in {\mathbb {Q}}(q^{\frac{1}{2}},t^{\frac{1}{2}})$. Thus, we obtained the vanishing loci of the Kac determinant $\mathrm {det}_n$. The prefactor $g_{N,n}(q,t)$ has been evaluated in [O]. $\quad \square$

As a corollary of Proposition E.6, Fact 2.10 follows.

Appendix F. Examples

1.1 F.1 Examples of

We present examples of the transition matrix from to the PBW-type basis .

Examples of $\alpha ^{(+)}_{\varvec{\lambda },\varvec{\mu }}$ in the case $N=1$:

(F.1)

(F.2)

Examples of $\alpha ^{(+)}_{\varvec{\lambda },\varvec{\mu }}$ in the case $N=2$:

(F.3)

(F.4)

Examples of $\alpha ^{(-)}_{\varvec{\lambda },\varvec{\mu }}$ in the case $N=2$:

(F.5)

(F.6)

Examples of $\alpha ^{(+)}_{\varvec{\lambda },\varvec{\mu }}$ in the case $N=3$:

(F.7)

Examples of $\alpha ^{(-)}_{\varvec{\lambda },\varvec{\mu }}$ in the case $N=3$:

(F.8)

1.2 F.2 Examples of matrix elements

In this section, we demonstrate how to calculate the matrix elements by the defining relation of $\mathcal {V}(w)$. Let us first explain it in general case. If $\varvec{\lambda }\ne (\emptyset ,\ldots , \emptyset )$, let $j=\min \{i|\lambda ^{(i)}\ne \emptyset \}$. The defining relation gives

(F.9)

The first term can be rewritten by matrix elements satisfying $|\varvec{\nu }|=|\varvec{\lambda }|-1$ (particularly, in the case $\lambda ^{(j)}_1-1<\lambda ^{(j)}_2$), and the second term can be expanded by vectors of level $|\varvec{\rho }|=|\varvec{\mu }|-\lambda ^{(j)}_{1}$ or $|\varvec{\mu }|-\lambda ^{(j)}_{1}+1$. (If $|\varvec{\mu }|-\lambda ^{(j)}_{1},|\varvec{\mu }|-\lambda ^{(j)}_{1}+1<0$, it means just 0 vectors.) If $\varvec{\lambda }=(\emptyset ,\ldots , \emptyset )$, moving negative modes $X^{(i)}_{-n}$ to the left side of $\mathcal {V}(w)$ by the defining relation, we have the expression in terms of Young diagrams of smaller size. In this way, the matrix elements can be inductively and uniquely determined.

The followings are examples.

1.2.1 In $N=1$ case

First, by the defining relation of $\mathcal {V}(w)$, we have

(F.10)

By , and , we get

(F.11)

Similarly, it is easily seen that

(F.12)

Moreover, we have

(F.13)

By the direct calculation of the free field expression, we have

(F.14)

(F.15)

By (F.14) and (F.15), we obtain

(F.16)

The matrix element is already calculated. Since and in this particular case, is factorized and corresponds to the Nekrasov factor (the right hand side of (4.2)):

(F.17)

(F.11), (F.12) and (F.17) are the simplest examples of our main theorem (Theorem 4.4).

We list other cases. Let us first prepare the formula for the action of the algebra $\mathsf {U}(N)$ on the PBW-type basis .

(F.18)

(F.19)

(F.20)

(F.21)

By using these relation, the matrix elements for larger Young diagrams are inductively determined as follows:

(F.22)

(F.23)

(F.24)

(F.25)

(F.26)

(F.27)

(F.28)

(F.29)

(F.30)

(F.31)

(F.32)

(F.33)

By combining these matrix elements and the examples of transition matrices $\alpha ^{(\pm )}_{\varvec{\lambda }, \varvec{\mu }}$ from to in the last subsection, we can check Theorem 4.4.

1.2.2 In $N=2$ case

If there is only one box in all Young diagrams, it is clear that

(F.34)

(F.35)

(F.36)

(F.37)

As in the $N=1$ case, we prepare the formula for the action of $X^{(i)}_n$.

(F.38)

where

$$\begin{aligned}&\chi ^{(1)}_{1,1}= \frac{(t-1) v_1 (-q v_1+q t v_1+v_1+t v_2)}{t},\end{aligned}$$

(F.39)

$$\begin{aligned}&\chi ^{(1)}_{1,2}=\frac{(t-1) \sqrt{\frac{q}{t}} v_2 \left( t v_1 q^2-v_1 q^2+t v_2 q^2-v_2 q^2-t^2 v_1 q+t v_1 q+v_1 q+v_2 q+t^2 v_1-t v_1\right) }{q t},\end{aligned}$$

(F.40)

$$\begin{aligned}&\chi ^{(1)}_{2,1}= \frac{(t-1) v_1 v_2 (-q v_1+q t v_1+v_1+t v_2)}{t},\end{aligned}$$

(F.41)

$$\begin{aligned}&\chi ^{(1)}_{2,2}=\frac{(t-1) v_1 v_2 \left( t v_2 q^2-v_2 q^2-t v_2 q+v_2 q+t^2 v_1+t^2 v_2\right) }{\sqrt{\frac{q}{t}} t^2},\end{aligned}$$

(F.42)

$$\begin{aligned}&\chi ^{(2)}_{1,1}=\frac{(t-1) v_1 v_2 \left( q^2 t v_1+q^2 t v_2-q^2 v_1-q^2 v_2-q t v_1-q t v_2+q v_1+q v_2+t^2v_1\right) }{t^2},\end{aligned}$$

(F.43)

$$\begin{aligned}&\chi ^{(2)}_{1,2}=\frac{q (t-1) v_1 v_2 (q t v_1+q t v_2-q v_1-q v_2-t v_1+v_1+v_2)}{t^2 \sqrt{\frac{q}{t}}},\end{aligned}$$

(F.44)

$$\begin{aligned}&\chi ^{(2)}_{2,1}=\frac{(t-1) v_1^2 v_2^2 \left( q^2 t-q^2+q t^2-2 q t+q+t\right) }{t^2},\end{aligned}$$

(F.45)

$$\begin{aligned}&\chi ^{(2)}_{2,2}=\frac{(t-1) v_1^2 v_2^2 \left( q^2 t-q^2+q t^2-2 q t+q+t\right) }{t^2 \sqrt{\frac{q}{t}}}. \end{aligned}$$

(F.46)

By these, the matrix elements can be written as

(F.47)

(F.48)

(F.49)

(F.50)

(F.51)

Furthermore, the direct calculation gives

(F.52)

(F.53)

Here,

$$\begin{aligned} \kappa ^{(1)}_{1,1}=\,&q^{-1} t^{-2}(t-1) (q^2 t u_1^2+q^2 t u_2^2+q^2 t u_1 u_2-q^2 u_1^2-q^2 u_2^2-q^2 u_1 u_2-q t^2 u_2^2 \end{aligned}$$

(F.54)

$$\begin{aligned}&-q t^2u_1 u_2+q t u_2^2+2 q t u_1 u_2+q u_1^2+q u_2^2+q u_1 u_2-t u_2^2-t u_1 u_2),\end{aligned}$$

(F.55)

$$\begin{aligned} \kappa ^{(1)}_{1,2}=\,&\frac{(t-1) u_2 (q t u_2-q u_2+t u_1+u_2)}{t^2 \sqrt{\frac{q}{t}}},\end{aligned}$$

(F.56)

$$\begin{aligned} \kappa ^{(1)}_{2,1}=\,&-t^{-3}(t-1) u_1 u_2 (-q^2 t u_1-q^2 t u_2+q^2 u_1+q^2 u_2+q t^2 u_2\end{aligned}$$

(F.57)

$$\begin{aligned}&-q t u_2-q u_1-q u_2-t^2 u_2+tu_2),\end{aligned}$$

(F.58)

$$\begin{aligned} \kappa ^{(1)}_{2,2}=\,&\frac{(t-1) u_1 u_2 \sqrt{\frac{q}{t}} (q t u_2-q u_2+t u_1+u_2)}{t^2},\end{aligned}$$

(F.59)

$$\begin{aligned} \kappa ^{(2)}_{1,1}=\,&q^{-1} t^{-3}(t-1) u_1 u_2 (q^3 t u_1+q^3 t u_2\nonumber \\&-q^3 u_1-q^3 u_2-q^2 t u_1-q^2 t u_2+q^2 u_1+q^2 u_2+q t^2u_1+q t^2 u_2-t^3 u_2),\end{aligned}$$

(F.60)

$$\begin{aligned} \kappa ^{(2)}_{1,2}=\,&\frac{(t-1) u_1 u_2 \left( q^2 t u_1+q^2 t u_2-q^2 u_1-q^2 u_2-q t u_1-q t u_2+q u_1+q u_2+t^2 u_2\right) }{t^3 \sqrt{\frac{q}{t}}},\end{aligned}$$

(F.61)

$$\begin{aligned} \kappa ^{(2)}_{2,1}=\,&\frac{q (t-1) u_1^2 u_2^2 \left( q^2 t-q^2+q t^2-2 q t+q+t\right) }{t^4},\end{aligned}$$

(F.62)

$$\begin{aligned} \kappa ^{(2)}_{2,2}=\,&\frac{(t-1) u_1^2 u_2^2 \sqrt{\frac{q}{t}} \left( q^2 t-q^2+q t^2-2 q t+q+t\right) }{t^3}, \end{aligned}$$

(F.63)

$$\begin{aligned} \zeta ^{(1)}_{1}=\,&\frac{(q-1) (t-1) \left( q u_1^2+q u_2 u_1+q u_2^2-t u_2 u_1\right) }{q t},\end{aligned}$$

(F.64)

$$\begin{aligned} \zeta ^{(1)}_{2}=\,&\frac{(q-1) q (t-1) u_1 u_2 (u_1+u_2)}{t^2},\end{aligned}$$

(F.65)

$$\begin{aligned} \zeta ^{(2)}_{1}=\,&\frac{(q-1) (t-1) u_1 u_2 (u_1+u_2)}{t},\end{aligned}$$

(F.66)

$$\begin{aligned} \zeta ^{(2)}_{2}=\,&\frac{(q-1) (t-1) u_1^2 u_2^2 (q+t)}{t^2}. \end{aligned}$$

(F.67)

Thus, we obtain

(F.68)

(F.69)

(F.70)

(F.71)

1.2.3 In $N=3$ case

Also in the representations of higher level, we can calculate the matrix elements similarly:

(F.72)

(F.73)

(F.74)

(F.75)

(F.76)

(F.77)

List of Notations

1.1 General notations

Especially, in some propositions in Sect. 3, and in Sect. 4, they are specialized as

The Nekrasov factor

Taki’s Flaming factor

1.2 Algebras and representations

PBW-type bases (Definition 2.9)

1.3 Vertex operators associated with generalized Macdonald Functions

$$\begin{aligned}&\text {Screening currents (Definition}~3.10)\\&\quad S^{(i)}(z) := \overbrace{1\otimes \cdots \otimes 1}^{i-1}\otimes \phi ^{\mathrm {sc}}(\gamma ^{i-1} z)\otimes \overbrace{1\otimes \cdots \otimes 1}^{N-i-1},\qquad i=1,\dots ,N-1,\\&\phi ^{\mathrm {sc}}(z):= \exp \left( -\sum _{n>0} \frac{1}{n}\frac{1-t^{n}}{1-q^n}\gamma ^{2n} a_{-n} z^n\right) \exp \left( \sum _{n>0} \frac{1}{n}\frac{1-t^{-n}}{1-q^{-n}}a_{n} z^{-n}\right) \\&\qquad \otimes \exp \left( \sum _{n>0} \frac{1}{n}\frac{1-t^{n}}{1-q^n}\gamma ^n a_{-n} z^n\right) \exp \left( -\sum _{n>0} \frac{1}{n}\frac{1-t^{-n}}{1-q^{-n}}\gamma ^{-n} a_{n} z^{-n}\right) . \end{aligned}$$

Shifted screening currents

$$\begin{aligned}&\widetilde{S}^{(k)} (z)=S^{(k)}(\gamma ^{-2k}t^{-1} z). \end{aligned}$$

Screened vertex operators (Definition 3.14)

$$\begin{aligned}&\Phi ^{(0)}(x)=:\exp \left( \sum _{n>0} \frac{1}{n}\frac{1-t^{n}}{1-q^n}a^{(1)}_{-n} u^n\right) \exp \left( \sum _{n>0} \frac{1}{n}\frac{1-\gamma ^{2n}t^{n}}{1-q^{-n}}t^{-n}a^{(1)}_{n} x^{-n}\right) \\&\quad \times \exp \left( \sum _{n>0} \frac{1}{n}\frac{1-\gamma ^{2n}}{1-q^{-n}} \sum _{j=2}^N\gamma ^{(j-1)n} a^{(j)}_{n} x^{-n}\right) :, \\&\Phi ^{(k)}(x):= \prod _{i=1}^k \frac{(q;q)_{\infty } (q/t;q)_{\infty }}{(\frac{q u_i}{u_{k+1}};q)_{\infty }(\frac{q u_{k+1}}{tu_{i}};q)_{\infty }} \oint _{C} \prod _{i=1}^k \frac{dy_i}{2 \pi \sqrt{-1}y_i}\Phi ^{(0)}(x)\\&\quad \times S^{(1)}(y_1)\cdots S^{(k)}(y_k) g(x,y_1,\ldots ,y_k), \\&g(x,y_1,\ldots ,y_k):=\frac{\theta _q(tu_1 y_1/u_{k+1}x)}{\theta _q(t y_1/x)} \prod _{i=1}^{k-1}\frac{\theta _q(tu_{i+1} y_{i+1}/u_{k+1}y_i)}{\theta _q(t y_{i+1}/y_i)}. \end{aligned}$$

Cartan operator arising from the commutation relation between $\Phi ^{(k)}(x)$ and $X^{(i)}(z)$ (Lemma 3.19)

$$\begin{aligned}&\Psi ^+(z):= \exp \left( \sum \frac{1}{n}(1-\gamma ^{2n}) \sum _{j=1}^N\gamma ^{(j-1)n} a^{(j)}_n z^{-n}\right) . \end{aligned}$$

Composition of screened vertex operators (Definition 3.25)

$$\begin{aligned} V^{(\varvec{n})}(x_1,\ldots ,x_{|\varvec{n}|})&= \Phi ^{(0)}(x_1)\cdots \Phi ^{(0)}(x_{n_{1}}) \Phi ^{(1)}(x_{n_1+1})\cdots \Phi ^{(1)}(x_{n_{1}+n_{2}})\cdots \\&\quad \cdots \Phi ^{(N-1)}(x_{[N, 1]_{\varvec{n}}})\cdots \Phi ^{(N-1)}(x_{|\varvec{n}|}). \end{aligned}$$

1.4 Symmetric functions and vectors in the N-fold tensor Fock spaces

Generalized Macdonald functions (eigenfunctions of $X^{(1)}_0$) (Theorem 3.26)

Integral form (Definition 3.37)

Factor arising from the application of Rumanujan’s ${_1}\psi _1$ summation formula (Proposition 3.27)

1.5 Factors in the Macdonald functions and hypergeometric series

The Macdonald functions (Definition 3.21)

Kajihara and Noumi’s multiple basic hypergeometric series (Definition 4.5)

Factors arising in the transformation formula from $p_{n+m}$ to another series containg $p_m$ as inner summation (Definition 4.7)

where n, m are nonnegative integers and $\mu =(\mu _i)_{1\le i \le m} \in {\mathbb {Z}}^m$.

1.6 Mukadé operator and its relatives

The defining relation of the vertex operator $\mathcal {V}(x) : \mathcal {F}_{\varvec{u}} \rightarrow \mathcal {F}_{\varvec{v}}$ (Definition 4.1)

$$\begin{aligned} \left( 1-\frac{x}{z}\right) X^{(i)}(z) \mathcal {V}(x) =\left( 1- (t/q)^i\frac{x}{z}\right) \mathcal {V}(x) X^{(i)}(z) \quad i \in \{1,2,\dots ,N\}. \end{aligned}$$

Realization of $\mathcal {V}(x)$ for the special case $v_i=t^{-n_i}u_i$ (Definition 4.14

$$\begin{aligned}&{\widetilde{V}}^{(\varvec{n})}(x) = \lim _{x_i \rightarrow t^{|\varvec{n}|-i} x} \prod _{1\le i< j\le |\varvec{n}|} \frac{(tx_j/x_i;q)_{\infty }}{(qx_j/tx_i;q)_{\infty }} V^{(\varvec{n})} (x_1,\ldots ,x_{|\varvec{n}|})A^{-1}_{(|\varvec{n}|)}(x),\\&A_{(r)}(x) = \exp \left( \sum _{n>0} \frac{(1-(q/t)^r)(1-t^{(1-r)n})t^{2r}}{n(1-q^n)(1-t^{-n})} \sum _{i=1}^{N} \gamma ^{(i-1)n}a^{(i)}_{n}x^{-n} \right) . \end{aligned}$$

Mukadé operators connected toward vertical and horizontal directions (Definition 5.8)

$$\begin{aligned}&{\mathcal {T}}^{V}(\varvec{u}, \varvec{v};w),\quad {\mathcal {T}}^{H}(\varvec{u}, \varvec{v};w) ,\quad \mathcal {T}^H_{\varvec{\lambda },\varvec{\mu }}(\varvec{u}, \varvec{v};w),\quad \mathcal {T}^V_{\varvec{\lambda },\varvec{\mu }}(\varvec{u}, \varvec{v};w). \end{aligned}$$

Mukadé operator specialized so that Young diagrams are restricted to only one row (Eq. A.8)

$$\begin{aligned} \widetilde{{\mathcal {T}}}_i(x)=\widetilde{{\mathcal {T}}}_i(\varvec{u}; x). \end{aligned}$$