1 Introduction and discussion

The study of defects can be used to characterize the behavior of physical theories and the related mathematical structures. In this paper we are interested in surface defects in four-dimensional supersymmetric gauge theories; namely, real codimension two submanifolds were a specific reduction of the gauge connection taking place. This kind of defects has been widely investigated in many contexts from various different perspectives. The study of the rôle of defects in the classification of the phases of gauge theories was pioneered by ’t Hooft [1]. Surface defects were introduced by Kronheimer and Mrowka [2, 3] in the study of Donaldson invariants, while their rôle in the context of geometric Langlands correspondence was emphasized in [4]. The correspondence with two-dimensional conformal field theories [5] prompted a systematic analysis of surface defects and highlighted their relevance for quantum integrable systems [6, 7] and for the study of isomonodromic deformations and Painlevé equations [8,9,10]. In this paper we introduce and study surface defects supporting nested instantons with respect to the parabolic reduction of the gauge group at the defect. These defects are engineered from a \(\mathrm{{D7/D3}}\) brane system on a local compact complex surface S. The brane engineering naturally leads to a description of these defects and their effective supersymmetric field theories in terms of moduli spaces of representations of quivers in the category of vector spaces, the objects being the branes and morphisms the open strings suspended among them. Supersymmetric partition functions of these systems provide conjectural formulae for topological invariants of these moduli spaces, more precisely, since these are generically not smooth, for virtual invariants of them.

The brane system we consider is \(\mathrm{{D7/D3}}\) on a local fourfold embedded in the ten dimensional IIB superstring supersymmetric background, the D3-branes wrapping the non-contractible cycle S. The D3-branes effective theory is the topologically twisted Vafa–Witten (VW) theory [11] with two extra chiral multiplets in the fundamental describing the \(\mathrm{{D7/D3}}\) open string sector. The \(\mathrm{{D7}}\) branes gauge theory is related to (equivariant) Donaldson–Thomas theory [12] on the fourfold. Actually, we consider these theories in a non-trivial \(\Omega \)-background corresponding to the equivariant parameters associated with rotations along the non-compact directions of the fourfold. This leads to a refinement of the above mentioned gauge theories. We focus on the case \(S=T^2\times {\mathcal {C}}\), the last being a Riemann surface with punctures \(\{p_i\}\). Surface operators of this four-dimensional gauge theory are real codimension two defects located at \(T^2\times \{p_i\}\).

The effective theory describing the dynamics of such surface defects is obtained in the limit of small area of \({\mathcal {C}}\) and turns out to be a quiver gauged linear sigma model which flows in the infrared to a nonlinear sigma model of maps from \(T^2\) to the moduli space of nested instantons. This is a generalization of the usual ADHM instanton moduli space, structured on the decomposition of the gauge connection at the surface defect. It is obtained from the usual ADHM instanton moduli space by implementing a suitable orbifold action which generates the fractional fluxes of the gauge field at the defect. The partition function of the \(\mathrm{{D7/D3}}\) effective theory computes the equivariant (virtual) elliptic genus of this moduli space in the presence of matter content dictated by the topology of \({\mathcal {C}}\), which, for genus g amounts to g-hypermultiplets in the adjoint representation. Their contribution is encoded in a bundle \({{\mathcal {V}}}_g\) over the moduli space of nested instantons. The general formula for the elliptic genus is (2.16) which, in the particular case \(r=1\) and \(k=1\), calculates the virtual elliptic genus of the bundle \({{\mathcal {V}}}_g\) over the nested Hilbert scheme of points on \({{\mathbb {C}}}^2\). The explicit combinatorial expression of (2.16) is given by (3.57) in terms of nested partitions.

We also study the circle reduction of this system, which leads to a T-dual D6/D2 quantum mechanics. In this case, we find that the generating function of the defects, obtained by summing over all possible decompositions of the connection at the puncture, or in other terms over all possible tails of the quiver, displays a very nice polynomial structure in the equivariant parameters.

The partition function of the D-brane system is computed in two ways. One, worked out in Sect. 3.1, makes use of superlocalization formulae [13] directly leading to a sum over fixed points with weights computed from the character of the torus action on the nested instanton moduli space. An alternative derivation is performed in Sect. 3.2, where the \(T^2\) partition function is evaluated via a higher dimensional contour integral à la [14]. This can be also prescribed via Jeffrey–Kirwan residue method [15, 16], as it was used in the study of \(\mathrm{{D1/D7}}\) BPS bound state counting on \({\mathbb {C}}^3\) in [17]. We remark that although the residue method is computationally more demanding, it has the advantage of allowing for the study of wall crossing among spaces with different stability conditions by changing the integration contour [18, 19].

When one considers a single D7 brane, the nested instanton moduli space reduces to the nested Hilbert scheme of points on \({\mathbb {C}}^2\). Our brane construction provides a conjectural description of this space as the moduli space of representations of the quiver considered in Sect. 2.6. Moreover, in this case the summation over the tails of the quiver gives rise to polynomials related to the modified Macdonald polynomials, and the whole partition function is related to the generating function introduced in [20] to describe the cohomology of character varieties. The analog result for the full \(T^2\) partition function gives rise to special combinations of elliptic functions which can be regarded as an elliptic lift of these polynomials. We display few examples in equations (3.96), (3.99), (3.102). These formulae should encode the elliptic cohomology of character varieties and can be viewed as an elliptic virtual refinement of the generating function of [20]. We remark that the D6/D2 quantum mechanical system and its relation with [20] was studied in [21] via a different approach based on topological string amplitudes on orbifold Calabi–Yau.

The relation with character varieties can be understood from the fact that Vafa–Witten theory on \(S=T^2\times {\mathcal {C}}\) is known to reduce in the small \({\mathcal {C}}\) limit to a GLSM on \(T^2\) with target space the Hitchin moduli space over \({\mathcal {C}}\) [22]. This in turn is homeomorphic [23] to the character variety of \({\mathcal {C}}\), namely the moduli space of representations of the first fundamental group of \({\mathcal {C}}\backslash \{p_i\}\) into \(GL_n({\mathbb {C}})\) with fixed semisimple conjugacy classes at the punctures.

There are some open questions to be discussed about the above construction. Actually, the 2d (2, 0) \(\mathrm{{D3/D7}}\) quiver gauge theory that we consider is anomalous, the \(\mathrm{{D3/D7}}\) open string modes breaking (2, 2) to (2, 0) and generating an R-symmetry anomaly. Indeed instantons in the D7 brane gauge theory are sourced from D3 branes. The mathematical counterpart is that Donaldson–Thomas (DT) theory on fourfold has positive virtual dimension and requires the insertion of observables to produce the appropriate measure on the moduli space [24, 25]. To cure this, we introduce new fields with opposite representations with respect to the gauge group and global symmetries. These are sources of the insertion of suitable observables which compensate the R-symmetry anomaly. Actually, the extra fields we consider can be thought as arising from coupling of D3-branes to \(\overline{{\mathrm{D7}}}\)-branes. It was recently conjectured [26] that \(\overline{{\mathrm{D7}}}\)/D7 system undergoes tachyon condensation leaving behind D3-branes. This proposal is a generalization of the known condensation [27] of \(\overline{{\mathrm{D5}}}\)/D5 into D3s. Indeed in our calculations we find that, at special values of the equivariant parameters, the contribution of the D3/\(\overline{{\mathrm{D7}}}\) and \(\mathrm{D3/D7}\) modes to the elliptic genus cancels out, in line with the above expectations. It would be extremely interesting to further analyze a possible application of our technique to the string field theoretic description of D-branes/anti D-branes annihilation.

The mathematical implication of all this is that DT theory on the local surface fourfold should reduce to VW theory on the complex surface S itself, the corresponding partition function providing conjectural formulae for VW invariants on S in the presence of surface defects. We aim to further investigate this reduction in the future and to analyze the elliptic genus of the nested instanton moduli space and in particular of the nested Hilbert scheme of points on toric surfaces. This can be obtained via gluing the contributions of the local patches [28,29,30]. Let underline that our computations concern a refined version of VW theory, a refinement being given by the mass m of the adjoint hypermultiplet. Therefore, by studying the limit at \(m\rightarrow \infty \), with appropriately rescaled gauge coupling, we reduce to pure twisted \({{\mathcal {N}}}=2\) gauge theory computing higher rank equivariant Donaldson invariants of S. Moreover, while in this paper we considered \(S=T^2\times {{\mathcal {C}}}\), non-trivial elliptic fibrations or other product geometries can be studied. In this way our approach could be used to generalize the results on Donaldson invariants of [31, 32]. The general modular properties of these generating functions are worth to be analyzed [11, 33].

The surface defects considered in this paper are directly related to Hitchin system with regular singularities. It would be obviously interesting to consider the case of irregular singularities, in particular the ones related to Argyres–Douglas points of gauge theories [9, 34], and investigate their rôle and contribution to the above mentioned differential invariants [35].

Moreover, the relation of our results to representation theory and quantum integrable systems should be explored, in particular investigating whether the cohomology of the nested instanton moduli space hosts representations of suitable infinite dimensional Lie algebra, generalizing the results of [36,37,38]. Also the characterization of the polynomials appearing in the quantum mechanical limit is to be worked out, by studying recurrence relations and/or difference equations they satisfy. This would possibly open a window on the relation with quantum integrable systems. For example, in [39, 40], the relation between \(\mathrm{{D1/D5}}\) systems on \({{\mathbb {P}}}^1\) and quantum Intermediate Long Wave hydrodynamics was studied, finding that the mirror of the associated GLSM provides the Bethe ansatz equations of the latter. Analogous relations between the mirror of the 2d comet-shaped quiver gauge theories and suitable integrable systems are worth to be explored. Finally, the F-theory uplift of our construction would help to study dualities of these defect gauge theories and to generalize them to other gauge groups.

The rest of the paper is structured in two main Sections. In the first one we provide the general brane setup and a detailed derivation of the comet-shaped quiver from D-branes on orbifolds. We then discuss the reduction to quiver quantum mechanics and the relation to character varieties. In the second, we perform explicit computations of the relevant partition functions and the relation with modified Macdonald polynomials of the reduced quantum mechanical quiver theory.

2 D-branes, geometry and quivers

2.1 Preliminaries

Let us start by discussing the geometric D-branes setup.

We consider a Type IIB supersymmetric general background built as the total space of a rank three complex vector bundle \(V^3_S\) on a complex surface S

$$\begin{aligned} X_5 = {{\mathrm{tot}}}\left( V^3_S\right) \end{aligned}$$
(2.1)

where supersymmetry requires the Calabi–Yau condition \({{\mathrm{det}}}V^3_S={{\mathcal {K}}}_S\), where \({{\mathcal {K}}}_S\) is the canonical bundle over S. To place a \(\mathrm{{D3{-}D7}}\) system in such a background, we assume that \(V^3_S\) has the following simplified structure

$$\begin{aligned} V^3_S={{\mathcal {K}}}_S\otimes {{\mathrm{det}}}^{-1}V^2_S\oplus V^2_S \end{aligned}$$

where the rank two bundle \(V^2_S\) is otherwise unconstrained.

Let us therefore consider the theory of N D3-branes wrapping the complex surface S in the background of r D7-branes along the local surface fourfold \({{\mathrm{tot}}}\left( V^2_S\right) \).

The low-energy dynamics of the N D3-branes can be obtained as usual by dimensional reduction of the \({{\mathcal {N}}}=1\) \(D=10\) supersymmetric Yang–Mills theory on \(X_5\) down to their world volume. This produces a topologically twisted version of the \({{\mathcal {N}}}=4\) \(D=4\) theory on S [41] whose boson content is given by the gauge connection \({{\mathcal {A}}}\), a section \(\Phi _{{\mathcal {L}}}\) of the line bundle \({{\mathcal {L}}}={{\mathcal {K}}}_S\otimes {{\mathrm{det}}}^{-1}V^2_S\) and a doublet \(\Phi _{V^2}\) which is a section of \(V^2_S\), the latter describing the transverse motion of the D3-branes in the ambient \(X_5\). All these fields are in the adjoint representation of the U(N) gauge group. The above setup reduces to the Vafa–Witten topologically twisted \({{\mathcal {N}}}=4\) \(D=4\) on S if the rank two vector bundle \(V^2_S={{\mathbb {C}}}^2\) is trivial and therefore \(X_5={{\mathrm{tot}}}\left( {{\mathcal {K}}}_S\right) \times {{\mathbb {C}}}^2\). In this case, the above construction indeed gives the gauge connection \({{\mathcal {A}}}\) on S, a complex (2, 0)-form \(\Phi _{S}\) valued in the fiber of \({{\mathcal {K}}}_S\) describing the transverse D3-branes motion within the local surface \(X_3={{\mathrm{tot}}}\left( {{\mathcal {K}}}_S\right) \), while the motion along the remaining \({{\mathbb {C}}}^2\) transverse directions is described by two other complex scalars \(B_i\), with \(i=1,2\).

The effect of the additional r background D7-branes on the D3-branes is kept into account by a further set of two complex scalars I and J in the bifundamental \(N\times {\bar{r}}\) and \(r\times {\bar{N}}\) of the gauge symmetry group U(N) and flavor global U(r) group. These are sections, respectively, of \({{\mathcal {O}}}_S\) and \({{\mathrm{det}}} V^2_S\) in general. This follows from the fact that these fields are in the positive chiral spinorial representation of the transverse SO(4) and are therefore sections of \({{\mathcal {S}}}_+ \sim \Lambda ^{(\mathrm{even},0)}(V^2_S)\), for S a Kahler surface.

The continuous symmetries of this geometric setup in the transverse directions to the D3-branes are the \(\left( {{\mathbb {C}}}^*\right) ^3\)-action on the \({{\mathbb {C}}}^3\) fiber of \(V_S^3\) with respective weights \((\epsilon _1,\epsilon _2,m)\). These are the global symmetries of the gauge theory on S which can be uses to define the relevant \(\Omega \)-background after turning on the relative background gauge fields. The parameter m introduces a mass for the adjoint hypermultiplet of the four-dimensional theory inducing the supersymmetry breaking from the \({{\mathcal {N}}}=4\) Vafa–Witten theory to its \({{\mathcal {N}}}=2^*\) refined version.

In the following, we will study the above general system in the case in which the complex surface is in the product form \(S=T^2\times {{\mathcal {C}}}\), where \({{\mathcal {C}}}\) is a Riemann surface and \(V^2_S\) is trivial. In this case, the canonical bundle over S reduces to the holomorphic cotangent bundle over \({{\mathcal {C}}}\) and

$$\begin{aligned} X_5={{\mathrm{tot}}}\left( T^*{{\mathcal {C}}}\right) \times T^2\times {{\mathbb {C}}}^2\, . \end{aligned}$$

In order to introduce surface defects in the gauge theory, we’re going to generalize the above setup to the case in which \({{\mathcal {C}}}\) is punctured at the points where the defects are located. More precisely, the parabolic reduction of the gauge bundle at the punctures is encoded in an orbibundle structure. The effective two-dimensional field theory describing the dynamics of the defect is obtained from the above setup in the chamber of small \({{\mathcal {C}}}\) volume leading to a quiver gauged linear sigma model describing the relevant open string modes. In the IR this reduces to a nonlinear sigma model of maps from \(T^2\) to the moduli space of representation of the quiver above.

2.2 D-branes on the orbicurve and defects

Let us now generalize the above setup to the case in which \({{\mathcal {C}}}\) is an orbicurve, that is, a Riemann surface with elliptic singular points. This means that the local geometry at some marked points \(\{P_\alpha \}\) of \({{\mathcal {C}}}\) is that of the \({{\mathbb {Z}}}_{s_\alpha }\) quotient of a disk D acted by \(z_\alpha \rightarrow \omega _\alpha z_\alpha \) with \(\omega _\alpha ^{s_\alpha }=1\).

Placing D-branes on an orbicurve consists in excising a regular cylinder out of the total space of the corresponding regular vector bundle and prescribing new local transition functions defining the lift of the discrete group action to the total space of the vector bundle. This operation extends the vector bundle to an orbibundle.

Let us therefore consider the geometry of the D-branes in the vicinity of a marked point P of order s with local coordinate z. The action on the D-brane Chan–Paton factors induces a modification of the gauge symmetry due to D-branes fractionalization [42]. Let \(\gamma _\ell \) be the number of D-branes in the \(\ell \)th sector, namely the one corresponding to the charge \(\ell \) representation \(z^\ell \) of \({{\mathbb {Z}}}_{s}\). This corresponds to prescribe the new transition function at the excised disk as

$$\begin{aligned} g_P=\bigoplus _{\ell =0}^{s-1} z^\ell \mathbf{1}_{\gamma _\ell } \end{aligned}$$

and, correspondingly the local behavior of the gauge connection as

$$\begin{aligned} {{\mathcal {A}}}_P= g_P^{-1} d g_P=\left( \frac{\mathrm{d}\, z}{z}\right) \bigoplus _{\ell =0}^{s-1} \ell \,\,\mathbf{1}_{\gamma _\ell } = \left( \frac{\mathrm{d}\, {\tilde{z}}}{{\tilde{z}}}\right) \bigoplus _{\ell =0}^{s-1} \frac{\ell }{s} \mathbf{1}_{\gamma _\ell }\, , \end{aligned}$$

where \({\tilde{z}}=z^{s}\). This finally induces the local prescription on the curvature \({{\mathcal {F}}}=\mathrm{d}{{\mathcal {A}}}\) as

$$\begin{aligned} {{\mathcal {F}}}_P/(2\pi )= \sqrt{-1}\delta ({\tilde{z}})\mathrm{d}{{\tilde{z}}}\wedge \mathrm{d}\bar{{\tilde{z}}}\bigoplus _{\ell =0}^{s-1} \left( \frac{\ell }{s}\right) \mathbf{1}_{\gamma _\ell } \end{aligned}$$

which implements the realization of the real codimension two defect in the four-dimensional gauge theory. Let us remark that from the algebraic geometry viewpoint this corresponds to study sheaves on root stacks, which is a natural framework were fractional Chern classes appear [43].

One can better describe the resulting gauge theory structure of the local D-brane configuration from the viewpoint of the geometry of the covering disk with local coordinate \({\tilde{z}}=z^{s}\) (Fig. 1).

Fig. 1
figure 1

The “brane cake” describing the covering structure the D3-branes on the local orbifold disk

In the s-covering of the quotient disk, the \(\gamma _\ell \) D-branes in the \(\ell \)th sector and their images span \(\ell \) Riemann sheets. As a consequence the \(\ell \)th Riemann sheet is spanned by an overall number of \(n_\ell =\sum _{\ell '=\ell }^{s-1} \gamma _{\ell '}\) D-branes. Let us notice that the outward of the quotient disk is joined to the rest of the Riemann surface by the first Riemann sheet which is consistently covered by all the \(n_0=\sum _{\ell '=0}^{s-1}\gamma _{\ell '}=N\) D-branes.

2.3 Two-dimensional quiver GLSM from the reduction to small \({{\mathcal {C}}}\) volume: bulk part

Let us consider now the reduction to small \({{\mathcal {C}}}\) volume of the system above. This leaves behind a gauge theory on the leftover \(T^2\) world volume whose spectrum can be computed by harmonic analysis. We denote by g the genus of \({{\mathcal {C}}}\).

Let us first discuss the reduction on a regular Riemann surface and then the more general situation in which \({{\mathcal {C}}}\) is an orbicurve.

The complex scalars I and J get simple dimensional reduction and stay scalars in the bifundamental, the gauge connection \({{\mathcal {A}}}\) on \(S={{\mathcal {C}}}\times T^2\) leaves behind the gauge connection A on \(T^2\) and g complex scalars in the adjoint, while other g complex scalars in the adjoint arise from the reduction of the transverse field \(\Phi _S\). These will be denoted as \(B_3^{(i)}\) and \(B_4^{(i)}\), where \(i=1,\ldots , g\).

The other two complex scalar fields in the adjoint, namely \(B_1\) and \(B_2\), get simply dimensionally reduced.

This field content results in the quiver in Fig. 2.

Fig. 2
figure 2

Quiver gauge theory arising from the compactification on \({\mathcal {C}}_{g,0}\)

The relations of this quiver can be read from the reduction of the F-term equations in “Appendix A” (A.1) and (A.3) by expanding in harmonic modes along the curve \({\mathcal {C}}\). More explicitly, the \(\Phi _S\) field and the component of the gauge connection \(A_{\mathcal {C}}\) along \({\mathcal {C}}\) give rise to the g hypers in the adjoint representation \((B_3^{(i)}, B_4^{(i)})\), where \(i=1,\ldots , g\), obeying the BPS equations

$$\begin{aligned}&[B_1,B_2] + IJ = 0\, , \, [B_3^i,B_4^j] = 0\nonumber \\&[B_1,B_3^i] = 0 \, , \, [B_1,B_4^i] = 0 \, , \, [B_2,B_3^i] = 0 \, , \, [B_2,B_4^i] =0 \nonumber \\&B_3^iI = 0 \, , \, JB_3^i = 0 \, , \, B_4^iI = 0 \, , \, JB_4^i = 0 \end{aligned}$$
(2.2)

The above equations are equivalent to g commuting copies of the ADHM equations for gauge theory with one adjoint hypermultiplet [13], as it can be shown by a simple squaring argument.

In the general \(\Omega \)-background the supersymmetry of the D3-brane system reduced on \(T^2\) is (2, 2), while the combined D3/D7-brane system reduced on \(T^2\) has (0, 2) supersymmetry due to the presence of the chiral fields I and J and the above field content, augmented by the relevant fermions, form the corresponding multiplets.

Let us underline that this theory itself suffers of a \(U(1)_R\)-symmetry anomaly due to its chiral unbalanced field content. This can be immediately understood from the fact that the D3-branes profile produces an instanton background in the D7-brane gauge theory inducing chiral symmetry breaking. From the mathematical viewpoint it is known that the Donaldson–Thomas theory on fourfold has positive virtual dimension which implies that one has to introduce observables matching the dimension counting. We propose that the suitable set of observables is given by a compensating sector of opposite charges—given by \({\bar{I}}\), \({\bar{J}}\) and other fields associated with the g-hypers to be specified later—which cancels the anomaly. This sector may be interpreted as a background antiD7-brane system.

2.4 Two-dimensional GLSM of the defect: the nested instanton quiver

When the curve \({{\mathcal {C}}}\) is extended to an orbicurve, at each orbifold point the gauge symmetry is reduced and further 2D degrees of freedom are present. These correspond to the open strings stretching between the twisted D-branes and, from the gauge theory viewpoint, to the degrees of freedom defining the codimension two defect prescribed by the singular behavior of the gauge curvature at the orbifold points.

To obtain the effective low-energy quiver description, we excise a disk around each puncture of \({{\mathcal {C}}}\) and discuss the local behavior of the D-branes system at the orbifold points computing the associated low-energy quiver gauge theory. We then glue back the disks to the bulk Riemann surface obtaining the full description of the gauge theory with defects reduced to two dimensions by the small \({{\mathcal {C}}}\)-volume limit. This procedure is pictorially described in Fig. 3.

Fig. 3
figure 3

Disk excision and gluing

The relevant open strings degrees of freedom can be inferred from the D-branes distribution as in Fig. 1. More precisely, see for example [44, 45], the Chan–Paton space of the D-brane system decomposes into irreducible representations \(R_\ell \) of the local discrete group \({{\mathbb {Z}}}_s\) as

$$\begin{aligned} {{\mathbb {V}}}= & {} \sum _{\ell =0}^{s-1} {{\mathbb {V}}}_\ell \otimes R_\ell \end{aligned}$$
(2.3)
$$\begin{aligned} {{\mathbb {W}}}= & {} \sum _{\ell =0}^{s-1} {{\mathbb {W}}}_\ell \otimes R_\ell \end{aligned}$$
(2.4)

where each of the D3- and D7-brane charged sectors is denoted as

$$\begin{aligned} {{\mathbb {V}}}_\ell ={{\mathbb {C}}}^{\gamma _\ell } \, , \, {{\mathbb {W}}}_\ell ={{\mathbb {C}}}^{\beta _\ell }\, . \end{aligned}$$
(2.5)

As depicted in Fig. 1, the \(\ell \)th Riemann sheet of the covering hosts a net number of \(n_j\equiv \sum _{\ell =j}^{s-1}\gamma _{\ell }\) D3-branes and of \(r_j\equiv \sum _{\ell =j}^{s-1}\beta _{\ell }\) D7-branes so that the open string degrees of freedom are represented as linear maps among the spaces

$$\begin{aligned} { V}_j= & {} \sum _{\ell =j}^{s-1} {{\mathbb {V}}}_\ell \end{aligned}$$
(2.6)
$$\begin{aligned} {W}_j= & {} \sum _{\ell =j}^{s-1} {{\mathbb {W}}}_\ell \end{aligned}$$
(2.7)

Let us now discuss the corresponding quiver gauge theory. This consists of a (0, 2) quiver gauge theory on \(T^2\) with gauge group \(\otimes _{j=0,\ldots , s-1} U(n_j)\), each node being coupled to two chiral multiplets in the adjoint \(B_1^j,B_2^j\in {{\mathrm{End}}} V_j\) and each pair of successive nodes by a chiral in the bifundamental \(F^j\in {{\mathrm{Hom}}}\left( V_j,V_{j+1}\right) \) for \(j=0,\ldots , s-1\). The \(\mathrm{D3{-}D7}\) open strings modes are described by the linear maps \(I^j\in {{\mathrm{Hom}}}\left( V_j,W_j\right) \) and \(J^j\in {{\mathrm{Hom}}}\left( W_j,V_j\right) \). Summarizing, the local \(\mathrm{{D3{-}D7}}\) system is effectively described by

$$\begin{aligned}&B_1^j,B_2^j\in {{\mathrm{End}}} V_j \, , F^j\in {{\mathrm{Hom}}}\left( V_j,V_{j+1}\right) \nonumber \\&I^j\in {{\mathrm{Hom}}}\left( V_j,W_j\right) \, {{\mathrm{and}}} \,J^j\in {{\mathrm{Hom}}}\left( W_j,V_j\right) \end{aligned}$$
(2.8)

As is shown in “Appendix A” these fields obey the relations

$$\begin{aligned}&[B_1^j,B_2^j]+I^jJ^j=0\, , \quad B_1^jF^j-F^jB_1^{j+1}=0\,\nonumber \\&\quad B_2^jF^j-F^jB_2^{j+1}=0\, , \quad J^jF^j=0. \end{aligned}$$
(2.9)

Therefore, the resulting quiver describing the local \(D3{-}D7\) system at the defect is given in Fig. 4.

Fig. 4
figure 4

Quiver gauge theory arising from the compactification on \({\mathcal {C}}_{g,1}\)

The moduli space \({\mathcal {N}}_{r,\lambda ,n,\mu }\) of its representations describes nested instantons. Indeed the n D3 branes realize an n-instanton profile for the U(r) D7 gauge fields, preserving the flag structure at the puncture. The partitions \(\lambda =(\lambda _1\ge \lambda _2\ge \cdots )\) of r and \(\mu =(\mu _1\ge \mu _2\ge \cdots )\) of n describe, respectively, the decomposition of the D7 and D3 Chan–Paton vector spaces into representations of the \({\mathbb {Z}}_s\) group. More precisely, as shown in Fig. 4, one gets the quiver of the flag manifold realized by the Chan–Paton vector spaces of the D3-branes \(V_{s-1}\subset V_{s-2}\subset \cdots \subset V_0\) with dimensions \(n_j= n_0 - \sum _{l=1}^{j} \mu _l\) framed by the D7 branes vector spaces with dimensions \(r_j=r_0 - \sum _{l=1}^{j} \lambda _l\). The heights of the columns of each partition is obtained from an ordering of the data of the dimensions vector spaces \(\beta _\ell \) and \(\gamma _\ell \) of (2.5). Indeed, these can be ordered by using Weyl symmetry of D3 and D7 branes gauge groups such that \(\gamma _0\ge \gamma _1\ge \cdots \ge \gamma _{s-1}\) and \(\beta _0\ge \beta _1\ge \cdots \ge \beta _{s-1}\).

The moduli space of nested instantons has a natural projection to the standard ADHM instanton moduli space \({\mathcal {M}}_{r,n}\)

$$\begin{aligned} \pi : {\mathcal {N}}_{r,\lambda ,n,\mu } \rightarrow {\mathcal {M}}_{r,n} \end{aligned}$$
(2.10)

which is realized by setting all the open string twisted sectors to be empty, namely by setting to zero all the fields \(F^j \, ,\, j=0,\ldots ,s-1\) and \((I^j,J^j,B_1^j,B_2^j)\) for \( j=1,\ldots ,s-1\).

2.5 Relation to other quiver defect theories

Some comments are in order regarding the quiver theory of the defect we obtain in our construction with respect to other quiver defect theories. The quiver we study is derived from a Dp/Dp+4 system via an orbifold action which affects a transverse direction to both the brane types. In this respect, it is different from the chain-saw quiver describing affine Laumon spaces [46], where the orbifold acts instead on the coordinates \(B_1,B_2\) describing the motion of Dp branes inside the Dp+4. This induces a different quiver with a different set of relations. A quiver which relates to the one in [46] can be obtained by considering a different specialization of the general geometric background for the \(\mathrm{{D3/D7}}\) system described in section 2.1. More precisely, one can consider \(T^2\times X_6\times {{\mathbb {C}}}_{\epsilon _1}\), where \(X_6={{\mathrm{tot}}}\left[ {{\mathcal {O}}}(p)\oplus {{\mathcal {O}}}(-p+2g-2)\right] _{{{\mathcal {C}}}_{g,k}}\) is the total space of a sum of two line bundles on the orbicurve such that the Calabi–Yau condition is satisfied. In such a geometry we can consider the D3-branes along \(T^2\times {{\mathcal {C}}}_{g,k}\) and the D7 say along \(T^2\times Y_4\times {{\mathbb {C}}}_{\epsilon _1}\), where \(Y_4={{\mathrm{tot}}}\left[ {{\mathcal {O}}}(p)\right] _{{{\mathcal {C}}}_{g,k}}\) and the fiber still hosts the torus action corresponding to the \(\epsilon _2\)-parameter of the Omega-background. For \(p>0\) in the vicinity of the orbifold points the geometry in the fiber direction is sensitive to the orbifold group.

Since the corresponding modes in the open string sector get twisted, the quiver changes. Instead of an adjoint multiplet per node, one gets a new bifundamental, while the fields \(J_i\) will now point from the gauge node \(n_i\) to the nearby framing node \(r_{i+1}\). The resulting local quiver at the defect is then the chain-saw quiver, as displayed in Fig. 5.

Fig. 5
figure 5

Chain-saw quiver

This can be also obtained from a \(\mathrm{{D1/D5}}\) system with both \(\mathrm{{D}}1\) and \(\mathrm{{D5}}\) wrapping \({\mathcal {C}}_{g,k}\) via a double T-duality along transverse directions to both.

Since on the other hand both quivers are describing the parabolic reduction of the gauge connection on a surface defect, it is conceivable to expect that a relation can be found between the associated partition functions at least in some limit or suitable parameterization. This could require non-trivial combinatorial identities on the partition functions themselves, similarly to what discussed in [47] concerning the relation between orbifold and vortex-like defects.

Moreover, when decoupling the D7 branes by setting \(I^j=0, J^j=0\), the description of the D3-branes at the defect leads to the quiver in Fig. 6 which describes a flag manifold with extra adjoint hypers at each node.

Fig. 6
figure 6

Quiver gauge theory for D3-branes at a single puncture

We notice that also the TSU[N] quivers for defects studied [48,49,50,51,52,53] are based on flag manifold quivers but display a different field content. It should be possible to compare the two kind of defect gauge theories in suitable limits by finding an appropriate dictionary.

2.6 Nested Hilbert scheme of points

The nested instanton moduli space is expected to reduce for a single D7-brane \(r=r_0=1\) to the moduli space of the nested Hilbert scheme of points on \({\mathbb {C}}^2\), \({{\mathrm{Hilb}}}^{n,\mu }({\mathbb {C}}^2)\). In this particular case the quiver described in the previous subsection reduces to the one of Fig. 7 with relations

$$\begin{aligned}&[B_1^0,B_2^0]+I^0J^0=0\, , \quad [B_1^j,B_2^j] =0\ \, , \quad j\ge 1 \nonumber \\&B_1^jF^j-F^jB_1^{j+1}=0\, \quad B_2^jF^j-F^jB_2^{j+1}=0\, , \quad J^0F^0=0. \end{aligned}$$
(2.11)
Fig. 7
figure 7

Nested Hilbert scheme quiver

The moduli space of representations of this quiver is expected to provide an explicit description of \({{\mathrm{Hilb}}}^{n,\mu }({\mathbb {C}}^2)\). This has been indeed proven for the particular case of two-step nested Hilbert scheme \(n_j=0\) for \(j\ge 2\) in [54], where it is also shown that this variety is smooth for \(n_1=1\). Indeed it is known that for \(n_1>1\) the two-step nested Hilbert scheme is singular. Moreover, nested Hilbert schemes with more than two steps are always singular, and a fortiori also the nested instanton moduli space. The \(\mathrm{{D3/D7}}\) partition functions we will evaluate via localization will then compute virtual invariants of these moduli spaces, since a perfect obstruction theory for them is expected to exist.

2.7 Comet-shaped quiver

Finally, the description of the \(\mathrm{{D3/D7}}\) system on the full geometry gives rise to the comet-shaped quiver in Fig. 8.

Fig. 8
figure 8

The comet-shaped quiver

This is obtained by gluing the nested instanton moduli quivers describing the decompositions of the branes at the defects to the bulk quiver in Fig. 2. The number of tails in the comet quiver is equal to the number of punctures of the Riemann surface, while their length is related to the flag structure due to the parabolic reduction of the connection at each puncture. All in all, the effective theory describing the \(\mathrm{{D3{-}D7}}\) system on \(T^2\) reduces to a GLSM with target space the total space of the bundle

$$\begin{aligned} {\mathcal {V}}_g \equiv \pi ^*\left( \left( T^*{\mathcal {M}}_{r,n}\right) ^{\oplus g} \otimes \left( \det {\mathcal {T}}\right) ^{1-g}\right) \end{aligned}$$
(2.12)

over the moduli space of nested instantons \({\mathcal {N}}_{r,{\underline{\lambda }},n,{\underline{\mu }}}\), where the collection of partitions \({\underline{\lambda }} =(\lambda ^1,\ldots ,\lambda ^k)\) and \({\underline{\mu }} =(\mu ^1,\ldots ,\mu ^k)\) describe the decomposition of D7 and D3 branes, respectively, under the cyclic groups \({\mathbb {Z}}_{s_i} \, , \, i=1,\ldots k\) acting at the punctures. The physical interpretation of the above bundle is the following: The first factor is simply the contribution of the g hypermultiplets in the adjoint representation of the bulk theory described in Sect. 2.3. Regarding the second factor, let us remark that the couplings of the \(\mathrm{{D3/D7}}\) brane system turns on a background line bundle describing the determinant bundle of the Dirac zero modes in the instanton background. This is given by the determinant of the tautological bundle \({\mathcal {T}}\) over \({\mathcal {M}}_{r,n}\). The power \((1-g)\) is due to the multiplicity of fermionic zero modes on the Riemann surface \({\mathcal {C}}\). In the limit of degeneration of the \(T^2\) to a circle this leads to a Chern–Simons interaction term for the resulting D2/D6 system. This term is essential in the comparison with results on character varieties and will be discussed in detail in Sect. 2.9, while in the next one we will briefly recall some basic definitions about character varieties that will be useful for the subsequent discussion.

2.8 Character varieties

Given a Riemann surface \({\mathcal {C}}\) of genus g with k punctures \(D=\sum _{i=1}^k p_i\), one defines the \(GL_n({\mathbb {C}})\) character variety as the moduli space of representations of the first fundamental group of \({\mathcal {C}}\backslash D\) into \(GL_n({\mathbb {C}})\)

$$\begin{aligned} {\mathcal {G}}_{\sigma } = \left\{ \rho \in {{\mathrm{Hom}}} \left( \pi _1\left( {\mathcal {C}}\backslash D\right) , GL_n({\mathbb {C}})\right) | \rho (\gamma _i) \in C_i \right\} / / PGL_n({\mathbb {C}}) \end{aligned}$$
(2.13)

where \(C_1,\ldots ,C_k\subset GL_n({\mathbb {C}})\) are semisimple conjugacy classes of type \(\sigma ^1,\ldots ,\sigma ^k\), namely the parts of the partition \(\sigma ^i\), \((\sigma ^i_1\ge \sigma ^i_2\ge \cdots )\), describe the multiplicities of the eigenvalues of any matrix in the conjugacy class \(C_i\).

When non-empty (2.13) is a smooth projective variety of dimension

$$\begin{aligned} d_\sigma = n^2 (2g-2+k) - \sum _{i,j} \left( \sigma _{\, j}^i\right) ^2 + 2 \end{aligned}$$

To describe the cohomology of (2.13), Hausel–Letellier–Rodriguez–Villegas [55] introduced the k punctures, genus g Cauchy function

$$\begin{aligned} \Omega (z,w) =\sum _{\sigma \in {\mathcal {P}}} {\mathcal {H}}_\sigma (z,w) \prod _{i=1}^k {\tilde{H}}^i_\sigma (\mathbf{x}_i; z^2,w^2) \end{aligned}$$
(2.14)

where \({\mathcal {P}}\) is the set of partitions, \( {\tilde{H}}^i_\sigma (\mathbf{x}_i; z^2,w^2)\) are refined Macdonald polynomials and

$$\begin{aligned} {\mathcal {H}}_\sigma (z,w)=\prod _{s\in \sigma }\frac{(z^{2a(s)+1} -w^{2l(s)+1})^{2g}}{(z^{2a(s)+2-w^{2l(s)}})(z^{2a(s)}-w^{2l(s)+2})}. \end{aligned}$$
(2.15)

where a(s), l(s) are, respectively, the arm and leg length of the s box of the Young diagram \(\sigma \) representing the partition (see Fig. 9). Equation (2.14) turns out to be the generating function of the cohomology polynomials of \(GL_n({\mathbb {C}})\) character varieties, summed over n.

Fig. 9
figure 9

Arm and leg length of a box in a Young diagram

Let us now outline the connection with the brane construction described in the previous subsections. The dynamics of D3-branes on the local surface S is refined Vafa–Witten theory. When \(S=T^2\times {\mathcal {C}}\), this reduces in the limit of small area of \({\mathcal {C}}\) to a gauged linear sigma model from \(T^2\) to Hitchin’s moduli space on \({\mathcal {C}}\) [22]. On the other hand, in [23] it was proved that (2.13) is homeomorphic to the moduli space of Higgs bundles with parabolic reduction on the divisor \(D=\sum _{i=1}^k p_i\). In presence of D7 branes, the non-perturbative effects on their dynamics are obtained by summing over the D3-branes partition functions. One then naturally obtains a generating function of the elliptic cohomology of \(GL_n({\mathbb {C}})\) character varieties. Summarizing the \(T^2\) partition function of the \(\mathrm{{D3{-}D7}}\) comet-shaped quiver reads

$$\begin{aligned} Z_{T^2}=\sum _{n}\sum _{{\underline{\mu }}\in {\mathcal {P}}(n)^k} (\mathbf{q }^{{\underline{\mu }}})^{\mathbf{r }} {{\mathrm{Ell}}}^{\text {vir}}\left( {\mathcal {N}}_{r,{\underline{\lambda }},n, {\underline{\mu }}}, {\mathcal {V}}_g \right) , \end{aligned}$$
(2.16)

with \((\mathbf{q }^{{\underline{\mu }}})^{\mathbf{r }}= \prod _{i=1}^k \prod _{\alpha =0}^{s_i-1} \left( q_{i,\, \alpha }^{|\mu ^i_{\alpha }|}\right) ^{r^i_{\alpha }}\) and

$$\begin{aligned} {{\mathrm{Ell}}}^{\text {vir}}\left( {\mathcal {N}}_{r,{\underline{\lambda }},n,{\underline{\mu }}}, {\mathcal {V}}_g \right) = { Ell} (T^{\text {vir}}{\mathcal {N}}){{\mathrm{ch}}}\left( {\mathcal {V}}_g \right) \cap \left[ {\mathcal {N}}_{r,{\underline{\lambda }},n,{\underline{\mu }}} \right] ^{\text {vir}}. \end{aligned}$$
(2.17)

For a single D7 brane \(\mathbf{r } =r_0 =1\), the above formulae can be understood as an elliptic virtual generalization of the generating function introduced by LHRV. Indeed, we will show in the following that in the limit of degeneration of \(T^2\) to a circle, one obtains LHRV formulae, or more precisely a virtual refinement of them.

2.9 Reduction to quantum mechanics, Chern–Simons term and LHRV formulae

In this subsection we summarize the reduction of the \(\mathrm{{D3/D7}}\) system on \(T^2\) to a quantum mechanical system in a T-dual picture. More precisely, if the two torus factorizes as \(T^2=S^1\times S^1\) and one of the two circles is taken to be very small, our D-brane system can be T-dualized along the small circle and reduced to a corresponding D2/D6 system on \({{\mathcal {C}}}\times S^1\). This corresponds to the quantum mechanics of the comet-shaped quiver with a Chern–Simons coupling, given by a phase factor \({\text{ e }}^{i m\int CS(A,F)}={\text{ e }}^{{\text{ i }}m\int {\text{ d }}x^\mu {\mathcal {A}}_\mu }\) so that the particle is coupled to an external vector potential. Let us briefly recall how this works in the standard ADHM case [56] in order to then generalize it to the nested instanton moduli space. The partition function is the equivariant index

$$\begin{aligned} Z_{S^1}=\sum _n q^{nr}{{\,\mathrm{Ind}\,}}\left( {\mathcal {M}}_{r,n},{\mathcal {L}}^{\otimes m}\right) , \end{aligned}$$
(2.18)

where \({\mathcal {L}}\) is the determinant line bundle \({\mathcal {L}}={{\,\mathrm{Det}\,}}\not {D}\), whose fiber on the space of connections \({\mathcal {A}}/{\mathcal {G}}\) is . By making use the ADHM construction for the moduli space of ASD connections, the n-dimensional vector space \(V_0\) is actually the space of fermionic zero modes. In order to compute the Chern–Simons level, we make use of the Atiyah–Singer index theorem for a vector bundle \(E\rightarrow M\)

$$\begin{aligned} {{\,\mathrm{Ind}\,}}(M,E)={{\,\mathrm{Ind}\,}}(\not {D})=\int _M{\hat{A}}(TM)\wedge {{\,\mathrm{ch}\,}}E, \end{aligned}$$
(2.19)

which gives the index of the Dirac operator twisted by E, ie \(D:S\otimes E\rightarrow S\otimes E\), S being the spin bundle over M. To compute the CS level in the case at hand one has to consider the geometric background \(S^1\times T^*{\mathcal {C}}\times {\mathbb {C}}^2\times {\mathbb {R}}\). Because of the twisting of the supersymmetric theory along \({\mathcal {C}}\), the \(\not {D}\) operator along \({\mathcal {C}}\) reduces to the \({\overline{{\partial }}}\) operator and the roof genus \({\hat{A}}(TM)\) to the Todd class. Thus, when we take the effective theory obtained by shrinking the size of \({\mathcal {C}}\), \({{\,\mathrm{Ind}\,}}({\overline{{\partial }}})_{{\mathcal {C}}}\) gives the multiplicity of the fermionic zero modes, according to the decomposition \(\Psi ^{(0)}=\psi ^{(0)}_{{\mathcal {C}}}\otimes \psi ^{(0)}_{{\mathbb {C}}^2}\). The index theorem along \({\mathcal {C}}\) reads

$$\begin{aligned} {{\,\mathrm{Ind}\,}}({\overline{{\partial }}})_{{\mathcal {C}}}=\int _{{\mathcal {C}}}{{\,\mathrm{Td}\,}}(T{\mathcal {C}})=1-g, \end{aligned}$$
(2.20)

which determines the level of the Chern–Simons interaction to be \(m=1-g\). Finally, the partition function is given by the following equivariant (virtual) index

$$\begin{aligned} Z_{ S^1}=\sum _{n}\sum _{\mu \in {\mathcal {P}}(n)}(\mathbf{q }^{\mu })^r{{\,\mathrm{Ind}\,}}\left( {\mathcal {N}}_{r,\lambda ,n,\mu },{{\,\mathrm{Det}\,}}(\not {D})^{\otimes (1-g)}\right) , \end{aligned}$$
(2.21)

where we use the notation \(\mathbf{q }^\mu =q_0^{n_0(\mu )}\cdots q_{s-1}^{n_{s-1}(\mu )}\) and

$$\begin{aligned} {{\,\mathrm{Ind}\,}}\left( {\mathcal {N}}_{r,\lambda ,n,\mu },{{\,\mathrm{Det}\,}}(\not {D})^{\otimes (1-g)}\right) ={\hat{A}}(T^{\text {vir}}{\mathcal {N}}){{\,\mathrm{ch}\,}}\left( {{\,\mathrm{Det}\,}}(\not {D})^{\otimes (1-g)}\right) \cap \left[ {\mathcal {N}}_{r,\lambda ,n,\mu } \right] ^{\text {vir}}.\nonumber \\ \end{aligned}$$
(2.22)

In the quiver representation of the nested instanton moduli space, the \(\not {D}\) operator on \({\mathbb {C}}^2\) appearing in the above equation is given by the pull-back of the tautological bundle \({\mathcal {T}}\) on the ADHM moduli space \({\mathcal {M}}_{r,n}\), so that its determinant line bundle coincides with the one of \({\mathcal {T}}\), which will be used in the equivariant localization formulae.

In the following Sect. 3 we will show that the above partition function, when computed for the particular case of the nested Hilbert scheme of points on \({\mathbb {C}}^2\), gives a virtual generalization of LHRV formulae and reduces precisely to them when the nested Hilbert scheme is smooth. Let us remark that the quantum mechanical system of the nested Hilbert scheme of points and its relation with LHRV formulae has been studied in [21] via a different approach based on topological string amplitudes on orbifold Calabi–Yau.

3 Partition functions

In this section we proceed to the evaluation of the partition function of the effective quiver gauge theories of the \(\mathrm{{D3/D7}}\)-system discussed in the previous section in the limit of small volume of the wrapped curve \({{\mathcal {C}}}\). This is performed by making use of supersymmetric localization which is a version of equivariant localization formulae [57] for super-manifolds which allows a generalization to supersymmetric path integrals in quantum field theories. The only configurations contributing to the latter are the fixed loci of the supersymmetry transformations. When these are isolated points, the path integral reduces to a sum over them weighted by one-loop super-determinants of the tangent bundle T at those points, that is,

$$\begin{aligned} \sum _{x\in {\{FP\}}} \frac{e^{-S(x)}}{\mathrm{Sdet} T_x} \end{aligned}$$
(3.1)

where \(\{FP\}\) is the set of fixed points, S(x) is the value of the action at \(x\in \{FP\}\) and \(T_x=T\vert _x\) is the restriction of T at x.

In the following we will implement the above computational scheme by calculating the above data for the relevant quiver gauge theories on \(T^2\). We will first focus on the contribution of a single defect on the sphere encoding the parabolic reduction of the connection at a given point, which is described by a single legged quiver. Then, we will consider the case of higher genus Riemann surface and combine all the contributions in the comet-shaped quiver theory partition function.

3.1 Contribution of a single surface defect on the sphere

3.1.1 Field content and superpotential

The matter content of the GLSM we are interested in is the one summarized in Table 1, where \(G=U(n_0)\times U(n_1)\times \cdots \times U(n_{s-1})\) and \(\Box _i\) denotes the Young diagram corresponding to the fundamental representation of \(U(n_i)\).

Table 1 Field content for quiver in Fig. 10

The relations satisfied by the quiver GLSM are enforced by the superpotential \({\mathcal {W}}\) in (3.2).

$$\begin{aligned} \begin{aligned} {\mathcal {W}}&={{\,\mathrm{Tr}\,}}_{0}\left[ \chi _0([B_1^0,B_2^0]+IJ)\right] +\sum _{i=1}^N{{\,\mathrm{Tr}\,}}_{i}\left[ \chi _i[B_1^i,B_2^i]+\chi _i^{B_1}(B_1^{i-1}F^i-F^iB_1^i) \right. \\&\quad \left. +\chi _i^{B_2}(B_2^{i-1}F^i-F^iB_2^i)\right] +\chi _{JF}JF^1. \end{aligned} \end{aligned}$$
(3.2)

Let us notice that, as we already pointed out, the locus cut out by \({\mathcal {W}}\) through the D-term equations is overdetermined. Thus, we still have to introduce \(s-1\) additional chiral fields \(Q_i,\ i=1,\dots ,{s-1}\) taking care of the relations over the constraints. These additional fields will transform in the \(\overline{\Box }_i\otimes \Box _{i-1}\) representation of \(U(n_i)\times U(n_{i-1})\). We will assign them R-charge 2q and they will be charged under the \(U(1)^2\) flavor symmetry with charge (1, 1). The relations over the constraints induced by these chirals are

$$\begin{aligned} \begin{aligned} 0&=[B_1^{i-1},B_2^{i-1}]F^i+B_2^{i-1}(B_1^{i-1}F^i-F^iB_1^i)-(B_1^{i-1}F^i-F^iB_1^i) B_2^i\\&\quad +(B_2^{i-1}F^i-F^iB_2^i)B_1^i-B_1^{i-1}(B_2^{i-1}F^i-F^iB_2^i)-F^i[B_1^i,B_2^i], \end{aligned} \end{aligned}$$
(3.3)

when \(i>1\), while

$$\begin{aligned} \begin{aligned} 0&=([B_1^{0},B_2^{0}]+IJ)F^1+B_2^{0}(B_1^{0}F^1-F^1B_1^1)-(B_1^{0}F^1-F^1B_1^1)B_2^1\\&\quad +(B_2^{0}F^1-F^1B_2^1)B_1^1-B_1^{0}(B_2^{0}F^1-F^1B_2^1)-I(JF^1)-F^1[B_1^1,B_2^1]\, \end{aligned} \end{aligned}$$
(3.4)

covers the remaining case \(i=1\).

The chiral supersymmetry transformations of the above fields are

$$\begin{aligned}&QI=\mu _I \, ,\quad Q\mu _I=D_AI+ \phi ^0 I-Ia \end{aligned}$$
(3.5)
$$\begin{aligned}&QJ=\mu _J \, ,\quad Q\mu _J=D_AJ - J\phi ^0 + a J-\epsilon J\end{aligned}$$
(3.6)
$$\begin{aligned}&QB^i_l=M^i_l \, , \quad QM^i_l = D_AB^i_l + [\phi ^i,B^i_l] -\epsilon _lB^i_l\end{aligned}$$
(3.7)
$$\begin{aligned}&Q\psi _F^i=F^i\, , \quad QF^i=D_A\psi _F^i-\phi ^i\psi _F^i+\psi _F^i\phi ^{i+1}\end{aligned}$$
(3.8)
$$\begin{aligned}&Q\chi _i=h_i\, , \quad Qh_i=D_A\chi _i +[\phi ^i, h_i] +\epsilon h_i\end{aligned}$$
(3.9)
$$\begin{aligned}&Q\chi _{JF}=h_{JF}\, , \quad Qh_{JF}=D_A\chi _{JF} + [\phi ^0,\chi _{JF}] + (\epsilon -a)\chi _{JF}\end{aligned}$$
(3.10)
$$\begin{aligned}&Q\chi _i^{B_l}=h_i^{B_l}\, , \quad Qh_i^{B_l}= D_A\chi _i^{B_l}+\phi ^i\chi _i^{B_l} -\chi _i^{B_l}\phi ^{i+1}+\epsilon _l\chi _i^{B_l}\end{aligned}$$
(3.11)
$$\begin{aligned}&Q\chi _{Q_i}=h_{Q_i}\, , \quad Qh_{Q_i}= D_A\chi _{Q_i}+\phi ^{i-1}\chi _{Q_i}-\chi _{Q_i}\phi ^i +\epsilon \chi _{Q_i}\end{aligned}$$
(3.12)
$$\begin{aligned}&Q{\bar{A}}=\eta \, , \quad Q\eta =F_A\, , \quad QA=0 \end{aligned}$$
(3.13)

where \((A,{\bar{A}})\) is the connection on \(T^2\) in holomorphic coordinates and \(F_A\) its curvature two-form, \(\epsilon _l\), \(l=1,2\) are the equivariant weights of the \(U(1)^2\) rotation group acting on \({\mathbb {C}}^2\) and \(\epsilon =\epsilon _1+\epsilon _2\). Moreover, \(\phi ^i, \, i=0,\ldots ,s-1\) are the zero modes of the A-connection implementing global \(U(n_i)\) gauge transformations of the ith-node. The fixed points of the above supersymmetry transformation impose that the connection \((A,{\bar{A}})\) is flat. Then, by a standard squaring argument one can show that the other fields must be constant so that the supersymmetry fixed locus reduces to the fixed locus of the \(U(1)^{(r+2)}\)-torus action on the nested instanton moduli space, where \(U(1)^r\) is the Cartan torus of the U(r) gauge group with equivariant parameters \(a_b, \, b=1,\ldots ,r\).

3.1.2 Anomaly and observables

As we already discussed at the end of Sect. 2.3, the (0, 2) \(\mathrm{{D3/D7}}\)-branes theory displays a \(U(1)_R\) anomaly whose compensation can be obtained via the insertion of suitable observables. To this end we introduce a sector of additional degrees of freedom \({\bar{I}}\) and \({\bar{J}}\) with opposite gauge global charges w.r.t. I and J which, once integrated out, produces the insertion of the observables. These will be properly taken into account in the following computations.

3.1.3 Fixed points

The characterization of the fixed locus of the torus action on the moduli space of nested instantons \({\mathcal {N}}(r,n_0,\dots ,n_{s-1})\simeq {\mathcal {N}}_{r,[r^1],n,\mu (n)}\) is most easily understood by describing it as the moduli space of (suitably defined) stable representations of the quiver in Fig. 10. In this setting we associate to the quiver in Fig. 10 the vector spaces W and \(V_i\), in addition to the space

$$\begin{aligned}&{\mathbb {X}}={{\,\mathrm{End}\,}}(V_0)^{\oplus 2}\oplus {{\,\mathrm{Hom}\,}}(V_0,W)\oplus {{\,\mathrm{Hom}\,}}(W,V_0)\\&\oplus \left[ \bigoplus _{i=1}^{s-1} \left( {{\,\mathrm{End}\,}}(V_i)^{\oplus 2}\oplus {{\,\mathrm{Hom}\,}}(V_i,V_i-1)\right) \right] \end{aligned}$$

of the morphisms of the quiver corresponding to the matter fields \(B_{1,2}^i\), \(F^i\), I and J. In this language, the quiver in Fig. 10 would be represented graphically as the one in Fig. 11.

Fig. 10
figure 10

Low-energy GLSM quiver in the case of \(g=0\), \(k=1\)

Fig. 11
figure 11

General representation of quiver in Fig. 10

On \({\mathbb {X}}\) we have a natural action of \({\mathcal {G}}=GL(V_0)\times \cdots \times GL(V_{s-1})\), which preserves the subscheme \({\mathbb {X}}_0\) of those points satisfying the relations (2.11). Then, given a framed representation \((W,V_0,\dots ,V_{s-1},X)\), \(X\in {\mathbb {X}}_0\) of the quiver in Fig. 11, one can prove that there is a suitable definition of stability such that, in a particular chamber of the parameters at play, semi-stability is equivalent to stability (also as a GIT quotient), so that it makes sense to talk about the moduli space of stable framed representations of the quiver in Fig. 11 without any further specification. This space will be denoted by \({\mathcal {N}}(r,n_0,\dots ,n_{s-1}):={\mathbb {X}}_0/\!\!/_\chi {\mathcal {G}}\), for some suitable choice of an algebraic character \(\chi \) of \({\mathcal {G}}\).

By means of this construction one can show that there is a sum decomposition \(V_0=V_i\oplus {\tilde{V}}_i\) and \(V_i=V_{i+1}\oplus {\hat{V}}_{i+1}\), such that \({\tilde{V}}_i={\hat{V}}_i\oplus {\tilde{V}}_{i-1}\). This splitting also induces the following block matrix decomposition of the morphisms \(B_{1,2}^0\), I and J in (3.14),

$$\begin{aligned} B_1^0=\begin{pmatrix} B_1^i &{} B_1^{'i}\\ 0 &{} {\tilde{B}}_1^i \end{pmatrix},\qquad B_2^0=\begin{pmatrix} B_2^i &{} B_2^{'i}\\ 0 &{} {\tilde{B}}_2^i \end{pmatrix},\qquad I=\begin{pmatrix} I^{'i}\\ {\tilde{I}}^i \end{pmatrix},\qquad J=\begin{pmatrix} 0&{\tilde{J}}^i \end{pmatrix}.\qquad \end{aligned}$$
(3.14)

such that \((W,{\tilde{V}}_i,{\tilde{B}}_1^i,{\tilde{B}}_2^i,{\tilde{I}}^i,{\tilde{J}}^i)\) is a stable ADHM datum.

Once an equivariant action of a torus \(T\curvearrowright {\mathcal {N}}(r,n_0,\dots ,n_{s-1})\) is introduced in the natural way suggested by the SUSY construction of the quiver in Fig. 10, the previous observations make it possible to characterize the T-fixed locus of \({\mathcal {N}}(r,n_0,\dots ,n_{s-1})\) in terms of those of some moduli spaces of stable ADHM data. This is all summarized in the following proposition

Proposition 1

The T-fixed locus \(\left[ {\mathcal {N}}(2,3,2,1)\right] ^T\) of \({\mathcal {N}}(r,n_0,\dots ,n_{s-1})\) is described by s-tuples of nested colored partitions \(\varvec{\mu }_1\subseteq \cdots \subseteq \varvec{\mu }_{s-1}\subseteq \varvec{\mu }_0\), with \(|\varvec{\mu }_0|=n_0\) and \(|\varvec{\mu }_{i>0}|=n_0-n_i\).

Example 1

As an example, consider the moduli space \({\mathcal {N}}(2,3,2,1)\). Its fixed point locus will be described by the following couples of nested partitions,

$$\begin{aligned} \left[ {\mathcal {N}}(2,3,2,1)\right] ^T\leftrightarrow \left\{ \begin{aligned}&\left( 1^1,2^1,3^1;\emptyset \right) ,\left( 1^1,2^1,2^1;1^1\right) ,\left( 1^1,2^1;1^1,1^1,1^1\right) ,\left( 1^1,1^1,2^1;1^1,1^1\right) ,\\&\left( 1^1;1^1,2^1,2^1\right) ,\left( 1^1,1^1,1^1;1^1,2^1\right) ,\left( 1^1,1^1;1^1,1^1,2^1\right) ,\left( \emptyset ,1^1,2^1,3^1\right) ,\\&\left( 1^1,2^1,2^11^1;\emptyset \right) ,\left( \emptyset ;1^1,2^1,2^11^1\right) ,\left( 1^1,1^2,2^11^1;\emptyset \right) ,\left( \emptyset ;1^1,1^2,2^11^1\right) ,\\&\left( 1^1,1^2,1^3;\emptyset \right) ,\left( 1^1,1^2,1^2;1^1\right) ,\left( 1^1,1^2;1^1,1^1\right) ,\left( 1^1,1^1,1^2;1^1,1^1\right) ,\\&\left( 1^1;1^1,1^2,1^2\right) ,\left( 1^1,1^1,1^1;1^1,1^2\right) ,\left( 1^1,1^1;1^1,1^1,1^2\right) ,\left( \emptyset ;1^11^21^3\right) , \end{aligned}\right. \end{aligned}$$

where each term on the rhs has to be interpreted as a couple of nested partitions, e.g.

The notation we use for a partition \(\mu \in {\mathcal {P}}\) is descriptive of its corresponding Young diagram in the following sense: \([\mu _1^{i_1}\cdots \mu _j^{i_j}\cdots ]\) denotes the partition

$$\begin{aligned} {\mathcal {P}}\ni [\mu _1^{i_1}\cdots \mu _j^{i_j}\cdots ] =(\underbrace{\mu _1,\dots ,\mu _1}_{i_1},\dots ,\underbrace{\mu _j,\dots , \mu _j}_{i_j},\dots ), \end{aligned}$$

or, in other words, \(i_j\) counts the number of rows of length \(\mu _j\) stacked one over the other.

3.1.4 Character computation

The super determinant weighting the contribution of each fixed point can be computed from the character decomposition of the torus action on the (virtual) tangent space:

$$\begin{aligned} \begin{aligned} T_Z^{\text {vir}}{\mathcal {N}}(r,n_0,\ldots ,n_{s-1})&={{\,\mathrm{End}\,}}(V_0) \otimes (Q-1-\Lambda ^2Q)+{{\,\mathrm{Hom}\,}}(W,V_0)+{{\,\mathrm{Hom}\,}}(V_0,W)\otimes \Lambda ^2Q+\\&\quad -{{\,\mathrm{Hom}\,}}(V_1,W)\otimes \Lambda ^2Q+\\&\quad +\sum _{\ell =1}^{s-1}\left[ {{\,\mathrm{End}\,}}(V_{\ell })-{{\,\mathrm{Hom}\,}}(V_{\ell },V_{\ell -1})\right] \otimes (Q-1-\Lambda ^2Q) \end{aligned} \end{aligned}$$
(3.15)

where Q denotes the representation \(T_1+T_2\) in the representation ring R(T). In the previous presentation of the virtual tangent space, the first line accounts for the standard ADHM quiver \((B^0_1,B^0_2, I,J)\) and their constraints, the second line for the constraint \(JF^1=0\) and the third line for the maps in the tail, their constraints and the relations among them.

By decomposing the vector spaces \(V_i\) in terms of characters of the torus action \(T\curvearrowright {\mathcal {N}}_{r,[r^1],n,\mu }\) we can then study the character decomposition of the virtual tangent space to the moduli space of nested instantons and obtain

$$\begin{aligned} \begin{aligned} T_Z^{\text {vir}}{\mathcal {N}}_{r,[r^1],n,\mu }&=T_{{\tilde{Z}}}{\mathcal {M}}_{r,n_0} +\sum _{a,b=1}^r\sum _{i=1}^{M_0^{(a)}}\sum _{j=1}^{N_0^{(b)}}R_bR_a^{-1} \left( T_1^{i-\mu _{1,j}^{(b)}}-T_1^{i}\right) \left( T_2^{-j+\mu _{1,i}^{(a)'}+1} +\right. \\&\quad \left. -T_2^{-j+\mu _{0,i}^{(a)'}+1}\right) -\sum _{i=1}^{M_0^{(a)}} \sum _{j=1}^{\mu _{0,i}^{(a)'}-\mu _{1,i}^{(a)'}}T_1^iT_2^{j+\mu _{1,i}^{(a)'}}+\\&\quad +\sum _{k=2}^{s-1}\left[ \sum _{a,b=1}^r\sum _{i=1}^{M_0^{(a)}}\sum _{j=1}^{N_0^{(b)}} R_bR_a^{-1}\left( T_1^{i-\mu _{k,j}^{(b)}}-T_1^{i-\mu _{k-1,j}^{(b)}}\right) \right. \\&\qquad \left. \left( T_2^{-j+\mu _{k,i}^{(a)'}+1}-T_2^{-j+\mu _{0,i}^{(a)'}+1}\right) \right] +(s-1)(T_1T_2), \end{aligned} \end{aligned}$$
(3.16)

where the fixed point Z is to be identified with a choice of a sequence of colored nested partitions \(\varvec{\mu }_1\subseteq \varvec{\mu }_{N-1}\subseteq \cdots \subseteq \varvec{\mu }_{s-1}\subseteq \varvec{\mu }_0\), as in proposition 1, \({\tilde{Z}}\leftrightarrow \varvec{\mu }_0\), \(\mu '\) denotes the partition transposed to \(\mu \) and \(T_i\), \(R_a\) are the generators of the torus action of \(U(1)^{r+1}\). Let us also point out that the last term, namely \((s-1)(T_1T_2)\), has been added in order to take into account the over-counting in the relations \([B_1^{i},B_2^{i}]=0\) due to the commutator being automatically traceless.

3.1.5 Determinants

Having the character decomposition of the virtual tangent space to the moduli space of nested instantons enables us to easily compute the 2d \({\mathcal {N}}=(0,2)\) partition functions of the low-energy GLSM of Sect. 2.4 in terms of the eigenvalues of the torus action, which we will do in the particular case of \(r=1\) for the sake of simplicity. The partition function we want to compute on the sphere \({\mathcal {C}}_0=S^2\) with 1 marked point will take the form

$$\begin{aligned} {\mathcal {Z}}^{\mathrm{ell}}_1(S^2;q_0,\dots ,q_{s-1})=\sum _{\mu _1\subseteq \cdots \subseteq \mu _0} q_0^{|\mu _0|}q_1^{|\mu _0\setminus \mu _1|}\cdots q_{s-1}^{|\mu _0\setminus \mu _{s-1}|}Z_{(\mu _0,\mu _1,\dots ,\mu _{s-1})}^{\mathrm{ell}},\nonumber \\ \end{aligned}$$
(3.17)

with \(|\mu _i\setminus \mu _j|=|\mu _i|-|\mu _j|\) denoting the number of boxes in the skew Young diagram \(Y_{\mu _i\setminus \mu _j}\), while \(Z^{\mathrm{ell}}_{(\mu _0,\dots ,\mu _{s-1})}\) is the contribution at a fixed instanton profile.

In particular, once we fix an instanton configuration by choosing a sequence of nested partitions \(\mu _1\subseteq \cdots \subseteq \mu _{s-1}\subseteq \mu _0\) we can write the torus partition function as

$$\begin{aligned} Z_{(\mu _0,\mu _1,\dots ,\mu _{s-1})}^{\mathrm{ell}}={\mathcal {L}}_{\mu _0}^{\mathrm{ell}}{\mathcal {N}}_{\mu _0}^{\mathrm{ell}}\overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}{\mathcal {T}}_{\mu _0,\mu _1}^{\mathrm{ell}}\overline{{\mathcal {T}}}_{\mu _0,\mu _1}^{\mathrm{ell}}{\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}}^{\mathrm{ell}}, \end{aligned}$$
(3.18)

where

$$\begin{aligned} {\mathcal {L}}^{\mathrm{ell}}_{\mu _0}&=\prod _{s\in Y_{\mu _0}}\exp \left[ -{{\,\mathrm{vol}\,}}(T^2)\left( \phi (s)-\xi \right) \right] , \end{aligned}$$
(3.19)
$$\begin{aligned} {\mathcal {N}}^{\mathrm{ell}}_{\mu _0}&=\prod _{s\in Y_{\mu _0}}\frac{1}{\theta _1(\tau |E(s))\theta _1(\tau |E(s)-\epsilon )},\end{aligned}$$
(3.20)
$$\begin{aligned} \overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}&=\prod _{s\in Y_{\mu _0\setminus \Box }}\theta _1(\tau |\phi (s)-{\tilde{a}})\theta _1(\tau |\phi (s)-{\tilde{a}}+\epsilon ),\end{aligned}$$
(3.21)
$$\begin{aligned} {\mathcal {T}}^{\mathrm{ell}}_{\mu _0,\mu _1}&=\prod _{i=1}^{M_0}\prod _{j=1}^{\mu _{0,i} -\mu _{1,i}}\theta _1(\tau |\epsilon _1i+\epsilon _2(j+\mu _{1,i}')),\end{aligned}$$
(3.22)
$$\begin{aligned} \overline{{\mathcal {T}}}^{\mathrm{ell}}_{\mu _0,\mu _1}&=\prod _{s\in Y_{\mu _0\setminus \mu _1}}\frac{1}{\theta _1(\tau |\phi (s)-{\tilde{a}}+\epsilon )},\end{aligned}$$
(3.23)
$$\begin{aligned} {\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}}&=\prod _{k=0}^{s-2}\left[ \prod _{i=1}^{M_0} \prod _{j=1}^{N_0}\frac{\theta _1(\tau |\epsilon _1(i+\mu _{k,j})+\epsilon _2 (\mu _{k+1,i}'-j+1))}{\theta _1(\tau |\epsilon _1(i-\mu _{k+1,j})+\epsilon _2 (\mu _{k+1,i}'-j+1))}\right. \nonumber \\&\left. \qquad \quad \prod _{i=1}^{M_0}\prod _{j=1}^{N_0}\frac{\theta _1(\tau | \epsilon _1(i-\mu _{k+1,j})+\epsilon _2(\mu _{0,i}'-j+1))}{\theta _1(\tau |\epsilon _1 (i-\mu _{k,j})+\epsilon _2(\mu _{0,i}'-j+1))}\right] , \end{aligned}$$
(3.24)

and for any box s in a Young diagram \(Y_\mu \) we defined \(\phi (s)\) to be the quantity (3.25)

$$\begin{aligned} \phi (s)=a+(i-1)\epsilon _1+(j-1)\epsilon _2, \end{aligned}$$
(3.25)

and

$$\begin{aligned} E(s)=-\epsilon _1a(s)+\epsilon _2(l(s)+1), \end{aligned}$$
(3.26)

with a(s) and l(s) being, respectively, the arm and leg length of s in \(Y_\mu \).

Notice that \({\mathcal {N}}_{\mu _0}^{\mathrm{ell}}\) is the elliptic analog of the Nekrasov partition function, while \(\overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}\) is its \(\overline{\text{ ADHM }}\) analog due to the \(\overline{{\mathrm{D7}}}\) coupling, and by \({\tilde{a}}\) we mean its Coulomb parameter. As in [58, 59] the \(\overline{{{\mathrm{D}}}7}\)-branes get stabilized in the presence of a B-field flux, so that their presence does not break supersymmetry and the low energy description of the D7/\(\overline{{{\mathrm{D}}}7}\) system is that of a U(N|N) gauge theory, [60,61,62]. Moreover, the contributions from the functions \({\mathcal {T}}^{\mathrm{ell}}_{\mu _0,\mu _1}\), \(\overline{{\mathcal {T}}}_{\mu _0,\mu _1}^{\mathrm{ell}}\) and \({\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}}\) altogether encode the contribution of the surface defect insertion. Finally, \({\mathcal {L}}^{\mathrm{ell}}_{\mu _0}\) encodes the CS-like term we discussed in Sects. 2.7 and 2.9 . This is interpreted as a CS-term contribution when the limit to QM is taken, and a 5d partition function on \({\mathbb {R}}^4\times S^1\) is retrieved. In any case, it comes from the coupling to a background connection on the determinant line bundle \({{\,\mathrm{Det}\,}}\not {D}\) encoding fermionic zero modes. This background connection is mirrored by the presence of \(\xi \) in (3.19), which is intended to be later specialized to \(\xi \rightarrow a\).

Because of the previous observations it is instructive to perform the summation over all the sequences of s nested partitions in two steps. First we sum over all the smaller partitions \(\mu _1\subseteq \cdots \subseteq \mu _{s-1}\subseteq \mu _0\) at fixed \(\mu _0\in {\mathcal {P}}\). It will prove useful for what we will do later to define the rational function \(P^{\mathrm{ell}}_{\mu _0}\) as in (3.27).

$$\begin{aligned} P^{\mathrm{ell}}_{\mu _0}=\sum _{\mu _1\subseteq \cdots \subseteq \mu _{s-1}} {\mathcal {T}}_{\mu _0,\mu _1}^{\mathrm{ell}}\overline{{\mathcal {T}}}_{\mu _0,\mu _1}^{\mathrm{ell}}{\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}}^{\mathrm{ell}}q_1^{|\mu _0\setminus \mu _1|}\cdots q_{s-1}^{|\mu _0\setminus \mu _{s-1}|}. \end{aligned}$$
(3.27)

Finally, by summing also over the \(\mu _0\) partitions we can rewrite the full partition function as in (3.28),

$$\begin{aligned} {\mathcal {Z}}^{\mathrm{ell}}_1(S^2;q_0,\dots ,q_{s-1})=\sum _{\mu _0}q_0^{|\mu _0|}{\mathcal {Y}}^{\mathrm{ell}}_{\mu _0}P^{\mathrm{ell}}_{\mu _0}, \end{aligned}$$
(3.28)

where we defined

$$\begin{aligned} {\mathcal {Y}}^{\mathrm{ell}}_{\mu _0}={\mathcal {L}}_{\mu _0}^{\mathrm{ell}}{\mathcal {N}}_{\mu _0}^{\mathrm{ell}}\overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}, \end{aligned}$$
(3.29)

and \(P^{\mathrm{ell}}_{\mu _0}\) are particular elliptic functions which can be regarded as an elliptic virtual uplift of modified Macdonald polynomials. The first few examples are listed in (3.96), (3.99), (3.102). As a useful remark, we want to point out that by taking the limit \(q_{i>0}\rightarrow 0\) we can effectively switch off the tail of the quiver, since \(P^{\mathrm{ell}}_{\mu _0}\xrightarrow {q_{i>0}\rightarrow 0}1\), and we recover the partition function on the sphere with one puncture of trivial holonomy, \({\mathcal {Z}}_0^{\mathrm{ell}}(S^2;q_0)\).

3.2 An alternative derivation: contour integral formulae

In this section we will be explicitly computing the partition functions of the low-energy theory coming from the D3/D7 system described in Sect. 2.4 by reducing the supersymmetric path integral to a contour integral via supersymmetric localization [15, 16].

The model we are interested in gives rise to a 2d \({\mathcal {N}}=(0,2)\) GLSM on \(T^2\). The mechanism of supersymmetry breaking from the maximal amount to \({\mathcal {N}}=(0,2)\) in the reduction to the low-energy theory leaves us with a matter content comprised of chiral fields corresponding to the morphisms in the representation theory of quiver in Fig. 7 in the category of vector spaces, and Fermi fields implementing the Lagrange multipliers in the superpotential. Let us first study the partition function for the quiver GLSM of Fig. 10, having fixed the numerical type of the quiver to \((1,n_0,\dots ,n_{s-1})\). In this case the localization formula is given by

$$\begin{aligned} Z_{T^2}=\frac{1}{(2\pi i)^N} \oint _C Z_{T^2,1-\text {loop}}(\tau ,z,{\textsf {x}}) \end{aligned}$$
(3.30)

where C is a real N-dimensional cycle in the moduli space of flat connections on \(T^2\) to be fixed with the Jeffrey–Kirwan prescription, \({\textsf {x}}\) denotes the collection of the (exponentiated) coordinates we are integrating over and

$$\begin{aligned} \begin{aligned}&{\hat{Z}}_{T^2,1-\text {loop}}(\tau ,z,{\textsf {x}})=\tilde{{\mathcal {Z}}} \left( \prod _{i\ne j}^{n_0}\frac{\theta _1(\tau |u_{ij}^0)\theta _1(\tau |u_{ij}^0-zq +\epsilon )}{\theta _1(\tau |u_{ij}^0+zq/2-\epsilon _1)\theta _1(\tau |u_{ij}^0+zq/2 -\epsilon _2)}\right. \\&\quad \left. \prod _{i=1}^{n_0}\frac{1}{\theta _1(\tau |u_i^0+z(q+p)/2-a) \theta _1(\tau |u_i^0-z(q-p)/2-a+\epsilon )}\right) \\&\quad \prod _{k=1}^{s-1}\left( \prod _{i\ne j}^{n_k}\frac{\theta _1(\tau |u_{ij}^k) \theta _1(\tau |u_{ij}^k-zq+\epsilon )}{\theta _1(\tau |u_{ij}^k+zq/2-\epsilon _1) \theta _1(\tau |u_{ij}^k+zq/2-\epsilon _2)}\right. \\&\quad \left. \prod _{i=1}^{n_k}\prod _{j=1}^{n_{k-1}}\frac{\theta _1(\tau |u_i^k-u_j^{k-1}-zq/2 +\epsilon _1)\theta _1(\tau |u_i^k-u_j^{k-1}-zq/2+\epsilon _2)}{\theta _1(\tau |u_{j}^{k-1} -u_i^{k})\theta _1(\tau |u_{j}^{k-1}-u_i^{k}+zq-\epsilon )}\right) \\&\quad \prod _{i=1}^{n_1}\theta _1(\tau |u_i^1+z(p-q)/2-a+\epsilon ), \end{aligned} \end{aligned}$$
(3.31)

with

$$\begin{aligned} \begin{aligned} \tilde{{\mathcal {Z}}}=\prod _{i=0}^{s-1}\left[ \frac{1}{n_i!} \left( \frac{2\pi \eta ^2(\tau )\theta _1(\tau |-zq+\epsilon )}{\theta _1(\tau |zq/2 -\epsilon _1)\theta _1(\tau |zq/2-\epsilon _2)}\right) \right] \frac{(\eta ^2(\tau ))^{n_0}}{({\text{ i }}\eta (\tau ))^{n_1}}. \end{aligned} \end{aligned}$$
(3.32)

As was already pointed out in Sect. 2.3, the coupling of the D3-branes to the D7-branes makes the theory anomalous. This chiral anomaly is encoded in the contributions dependent on the fields coupled to the framing, namely I and J, which break a chiral half of the original \({\mathcal {N}}=(2,2)\) supersymmetry. From the point of view of the localization formula this is most easily made manifest by studying the transformation properties of the integrand under shifts along the generators of the torus. Let us the recall that the Jacobi \(\theta _1(\tau |z)\) function is defined in terms of the exponentiated modular parameter \(q={\text{ e }}^{2\pi {\text{ i }}\tau }\), \(\mathfrak {I}\tau \ge 0\), and \(y={\text{ e }}^{2\pi {\text{ i }}z}\) as

$$\begin{aligned} \theta _1(\tau |z)=q^{1/8}y^{-1/2}(q,q)_\infty \theta (\tau |z), \end{aligned}$$

where \(\theta (\tau |z)=(y,q)_\infty (qy^{-1},q)_\infty \) and \((a,q)_\infty =\prod _{k=0}^\infty (1-aq^k)\) is the q-Pochhammer symbol. By this definition it is easy to see that the Jacobi function \(\theta _1(\tau |z)\) is odd in z, i.e. \(\theta _1(\tau |-z)=-\theta _1(\tau |z)\), and that it is quasi-periodic under shifts \(z\rightarrow z+a+b\tau \), \(a,b\in {\mathbb {Z}}\):

$$\begin{aligned} \theta _1(\tau |z+a+b\tau )=(-1)^{a+b}{\text{ e }}^{-2\pi {\text{ i }}bz}{\text{ e }}^{-{\text{ i }}\pi b^2\tau }\theta _1(\tau |z),\qquad \forall a,b\in {\mathbb {Z}}. \end{aligned}$$

The anomaly then comes from the fact the integrand is unbalanced in terms of the theta functions, exactly due to the presence of I and J (indeed their contribution appears in the second and last lines in Eq. (3.31)). The part of the 1-loop determinant coming from adjoint and bifundamental fields does not contribute to the gauge anomaly, as it comes from an \({\mathcal {N}}=(2,2)\) multiplet. As we already explained in Sect. 2.3, we take care of this anomaly by introducing extra Fermi fields \({\overline{I}}\) and \({\overline{J}}\), which we think can be interpreted as accounting for interactions with \(\overline{{\mathrm{D7}}}\)-branes. In this way we get that the \(T^2\) partition function is corrected by the presence of the \(\overline{{\mathrm{D7}}}\) as

$$\begin{aligned}&{\hat{Z}}_{T^2,1-\text {loop}}^{D3/D7/\overline{D7}}(\tau ,z,{\textsf {x}}) =\hat{{\mathcal {Z}}}\left( \prod _{i\ne j}^{n_0}\frac{\theta _1(\tau |u_{ij}^0)\theta _1 (\tau |u_{ij}^0-zq+\epsilon )}{\theta _1(\tau |u_{ij}^0+zq/2-\epsilon _1)\theta _1 (\tau |u_{ij}^0+zq/2-\epsilon _2)}\right. \nonumber \\&\quad \left. \prod _{i=1}^{n_0}\frac{\theta _1(\tau |u_i^0+zR_{{\overline{I}}}/2-{\overline{a}}) \theta _1(\tau |u_i^0+zR_{{\overline{J}}}/2-{\overline{a}}+\epsilon )}{\theta _1 (\tau |u_i^0+z(q+p)/2-a)\theta _1(\tau |u_i^0-z(q-p)/2-a+\epsilon )}\right) \nonumber \\&\quad \prod _{k=1}^{s-1}\left( \prod _{i\ne j}^{n_k}\frac{\theta _1(\tau |u_{ij}^k)\theta _1 (\tau |u_{ij}^k-zq+\epsilon )}{\theta _1(\tau |u_{ij}^k+zq/2-\epsilon _1)\theta _1 (\tau |u_{ij}^k+zq/2-\epsilon _2)}\right. \\&\quad \left. \prod _{i=1}^{n_k}\prod _{j=1}^{n_{k-1}}\frac{\theta _1(\tau |u_i^k-u_j^{k-1}-zq/2 +\epsilon _1)\theta _1(\tau |u_i^k-u_j^{k-1}-zq/2+\epsilon _2)}{\theta _1(\tau |u_{j}^{k-1} -u_i^{k})\theta _1(\tau |u_{j}^{k-1}-u_i^{k}+zq-\epsilon )}\right) \nonumber \\&\quad \prod _{i=1}^{n_1}\frac{\theta _1(\tau |u_i^1+z(p-q)/2-a+\epsilon )}{\theta _1(\tau |u_i^1+zR_{\overline{J}F}/2-{\overline{a}}+\epsilon )},\nonumber \end{aligned}$$
(3.33)

with

$$\begin{aligned} \begin{aligned} \hat{{\mathcal {Z}}}=(-1)^{n_1}\prod _{i=0}^{s-1}\left[ \frac{1}{n_i!} \left( \frac{2\pi \eta ^2(\tau )\theta _1(\tau |-zq+\epsilon )}{\theta _1(\tau |zq/2 -\epsilon _1)\theta _1(\tau |zq/2-\epsilon _2)}\right) ^{n_i}\right] . \end{aligned} \end{aligned}$$
(3.34)

Two observations are due here:

  1. 1.

    An appropriate choice of the R-charges \(R_{{\overline{I}}}\) and \(R_{{\overline{J}}}\) (which also determines the R-charge \(R_{{\overline{J}}F}\) relative to the multiplier for the constraint \(JF=0\)) makes it possible to overcome completely the anomaly issue in the integration variables and in the \(U(1)_R\) fugacity. However, asking for the double periodicity of the integrand forces us also to impose a constraint on the twisted masses a and \({\overline{a}}\), namely \({\tilde{a}}=a-{\overline{a}}\in {\mathbb {Z}}\). This condition is responsible for the fact that introducing the extra fields needed to cure the anomaly does not change the fixed point structure of the localization computation. The procedure we adopted has one additional beneficial side effect. In fact, even though the theory involving the \(\overline{{\mathrm{D7}}}\) branes is different from the one we started with, however it is still an interesting quantity, as it should compute a generating functions for insertions of observables, as it was proposed in the D8/\(\overline{{\mathrm{D8}}}\) case by Nekrasov in [63].

  2. 2.

    As for the second remark, it is interesting to study the QM limit (\(\tau \rightarrow {\text{ i }}\infty \)) of the partition function at hand. In fact when we shrink one \(S^1\) in \(T^2\) to a point, we can decouple the contribution of the \(\overline{\mathrm{D7}}\) branes by taking very large values of \({\overline{a}}\) and by then rescaling the relevant gauge coupling. By doing this we recover the 5d partition function one can independently compute on \({\mathbb {R}}^4\times S^1\), apart from an overall normalization factor. This will give us the equivariant Euler number of the nested Hilbert scheme of points on \({\mathbb {C}}^2\), possibly twisted by a power of the determinant line bundle of the Dirac \(\not {D}\) operator.

Now, in order to explicitly compute the partition function we need to remember that the Jacobi \(\theta _1(\tau |z)\) function does not have any pole; however, it has simple zeros on the lattice \(z\in {\mathbb {Z}}+\tau {\mathbb {Z}}\). Moreover, it is simple to verify that \(\theta (\tau |z)^{-1}\) has residue in \(z=\alpha +\beta \tau \) given by the following formula

$$\begin{aligned} \frac{1}{2\pi {\text{ i }}}\oint _{z=\alpha +\beta \tau }\frac{1}{\theta _1(\tau |z)} =\frac{(-1)^{\alpha +\beta }{\text{ e }}^{{\text{ i }}\beta ^2\tau }}{2\pi \eta ^3(\tau )}. \end{aligned}$$
(3.35)

In general a careful analysis of singularities would be needed in order to understand which poles are giving a non-vanishing contribution to the computation of the partition function on \(T^2\). In our particular case the poles contributing to the residue computation will be classified in terms of nested partitions \(\mu _1\subseteq \cdots \subseteq \mu _{s-1}\subseteq \mu _0\). In principle this result could be obtained via the systematic approach of Jeffrey–Kirwan. Here we follow an alternative procedure by giving a suitable imaginary part to the twisted masses (for example through the R-charges via a redefinition of the relevant parameters, as in [64]) and by closing the integration contour in the lower-half plane. In this particular setting we take care of redefining \(a,\,\epsilon _i\) in such a way that \(\mathfrak {I}a,\,\mathfrak {I}\epsilon _i<0\) and \(\mathfrak {I}a>\mathfrak {I}\epsilon \). By the requirement on the Cartan parameters of the D7-branes, namely \(a-{\overline{a}}\in {\mathbb {Z}}\), we also have \(\mathfrak {I}{\overline{a}}=\mathfrak {I}a<0\). It is sufficient to study the pole structure of the first two integrations (namely \(\{u_j^0\}\) and \(\{u_j^1\}\)) in (3.33), whose poles and zeros are schematically shown in Table 2.

Table 2 Poles and zeros for \(\{u_j^0\}\) and \(\{u_j^1\}\) in (3.33)

The integration over the \(\{u_j^0\}\) is standard, as it is has the same pole structure of the standard Nekrasov partition function [64, 65], and the poles contributing to the residue computation will be described by partitions \(\mu _0\). Each box in \(\mu _0\) will then encode the position of a pole for the first \(n_0\) integrations. As for the integrations over the \(\{u_j^1\}\) variables we first point out that the 1-loop determinant due to the \(\overline{{\mathrm{D7}}}\)-brane, as \(a-{\overline{a}}\in {\mathbb {Z}}\) and the corresponding pole falls out of the integration contour. In the same way also poles of the 1-loop determinant of \(Q_i\) give a vanishing contributions, because of one out of two different reasons: Either the singularity falls out of the integration contour, or its contribution is annihilated by a zero coming from the determinants of \(\chi _1^{B_i}\). Any pole that might fall outside the Young diagram associated with \(\mu _0\) must also be excluded from the computations, because of the flag structure of the quiver in Fig. 10. These considerations lead us to the classification of poles of the \(\{u_j^1\}\) integrations in terms of partitions as follows: By choosing the order of the integration to be \(u_1^1,u_2^1,\dots ,u_{n_1}^1\) poles are chosen by successively picking outer corners of \(Y_{\mu _0}\) so that the complement in \(Y_{\mu _0}\) is still a Young diagram corresponding to a partition \(\mu _1\), with \(|\mu _1|=n_0-n_1\). The procedure we just described is depicted in Fig. 12.

Fig. 12
figure 12

Procedure for picking poles of \(\{u_j^1\}\) from \(Y_{\mu _0}\)

Any successive integration is done in the same way, and the poles contributing to the integration are classified by sequences of nested partitions, as we discussed in Sect. 3.1.3.

Boxes in the skew partitions \(\mu _0\setminus \mu _j\) will denote positions for poles in the jth integration, according to the following rule: A box of \(Y_{\mu _0\setminus \mu _k}\) located at position (ij) inside \(Y_{\mu _0}\) (this is required by the nesting phenomenon) corresponds to the coordinate \(u_l^{(k)}=a+(i-1)\epsilon _1+(j-1)\epsilon _2\). One thing to be pointed out is that the assignment of a certain Young diagram configuration do in fact specify a particular pole only up to Weyl permutations of the coordinates: Because of this we choose a particular ordering of the coordinates and neglect the counting factor \((n_0!\cdots n_{s-1}!)^{-1}\) in \(\hat{{\mathcal {Z}}}\).

The partition function \(Z_{T^2}^{D3/D7/\overline{D7}}\) will then take the following form

$$\begin{aligned} \begin{aligned} Z_{T^2}^{D3/D7/\overline{D7}}=\hat{{\mathcal {Z}}}_{\text {res}} \sum _{\mu _1\subseteq \cdots \mu _{s-1}\subseteq \mu _0}\left( Z_{\mu _0} (\epsilon _1,\epsilon _2,{\overline{a}})Z^{JF}_{\mu _1,\mu _0}(\epsilon _1, \epsilon _2,{\overline{a}})\prod _{i=0}^{s-2}Z_{\mu _{i+1},\mu _i}(\epsilon _1, \epsilon _2,{\overline{a}})\right) , \end{aligned}\nonumber \\ \end{aligned}$$
(3.36)

with

$$\begin{aligned} \hat{{\mathcal {Z}}}_{\text {res}}&=(-1)^{n_1}\prod _{i=0}^{s-1} \left[ \frac{1}{n_i!}\left( \frac{\theta _1(\tau |-zq+\epsilon )}{\theta _1(\tau |zq/2-\epsilon _1)\theta _1(\tau |zq/2-\epsilon _2)\eta (\tau )}\right) ^{n_i} \right] \ \end{aligned}$$
(3.37)
$$\begin{aligned} Z_{\mu _0}(\epsilon _1,\epsilon _2,{\overline{a}})&=\prod _{s\in \mu _0\setminus \Box }\frac{\theta _1(\tau |\phi (s)-{\overline{a}})\theta _1(\tau |\phi (s)-{\overline{a}} +\epsilon )}{\theta _1(\tau |\phi (s))\theta _1(\tau |\phi (s)+\epsilon )}\cdot \nonumber \\&\quad \cdot \prod _{\begin{array}{c} s\ne s'\\ s,s'\in \mu _0 \end{array}} \left( \frac{\theta _1(\tau |\phi (s)-\phi (s'))}{\theta _1(\tau |\phi (s) -\phi (s')-\epsilon _1)}\frac{\theta _1(\tau |\phi (s)-\phi (s')+\epsilon )}{\theta _1(\tau |\phi (s)-\phi (s')-\epsilon _2)}\right) \end{aligned}$$
(3.38)
$$\begin{aligned} Z_{\mu _{k+1},\mu _k}(\epsilon _1,\epsilon _2,\overline{a})&=\prod _{\begin{array}{c} s\ne s'\\ s,s'\in \mu _0\setminus \mu _{k+1} \end{array}} \left( \frac{\theta _1(\tau |\phi (s)-\phi (s'))\theta _1(\tau |\phi (s) -\phi (s')+\epsilon )}{\theta _1(\tau |\phi (s)-\phi (s')-\epsilon _1)\theta _1(\tau |\phi (s) -\phi (s')-\epsilon _2)}\right) \cdot \nonumber \\&\quad \cdot \prod _{\begin{array}{c} s\in \mu _0\setminus \mu _{k+1}\\ s'\in \mu _0\setminus \mu _k \end{array}}\left( \frac{\theta _1(\tau |\phi (s)-\phi (s')+\epsilon _1) \theta _1(\tau |\phi (s)-\phi (s')+\epsilon _2)}{\theta _1(\tau |\phi (s')-\phi (s)) \theta _1(\tau |\phi (s')-\phi (s)-\epsilon )}\right) \end{aligned}$$
(3.39)
$$\begin{aligned} Z^{JF}_{\mu _1,\mu _0}(\epsilon _1,\epsilon _2,\overline{a})&=\prod _{\begin{array}{c} s\in \mu _0\setminus \mu _1\\ s'\in \mu _0 \end{array}}\frac{\theta _1(\tau |\phi (s)+\epsilon )}{\theta _1(\tau |\phi (s) -{\overline{a}}+\epsilon )} \end{aligned}$$
(3.40)

These formulae are to be compared with the contribution of a quiver with fixed numerical type to \({\mathcal {Z}}_1^{\mathrm{ell}}(S^2;q_0,\dots ,q_{s-1})\), in particular the contribution at each fixed point will be the same as \(Z_{(\mu _0,\dots ,\mu _{s-1})}^{\mathrm{ell}}\), which was defined in Sect. 3.1.5, but in principle one could use the same technique in order to compute partition functions in the more general case of a genus g Riemann surface \({\mathcal {C}}_g\).

3.3 General Riemann surfaces

When we switch from the genus 0 case to a generic Riemann surface \({\mathcal {C}}_g\) with 1 puncture, we are effectively turning on a matter bundle corresponding to the contribution of g adjoint hypermultiplets, and the quiver in Fig. 10 describing the GLSM we are studying gets modified into quiver in Fig. 13.

Fig. 13
figure 13

Low-energy GLSM quiver for a general \({\mathcal {C}}_{g,1}\)

This GLSM encodes the ADHM construction of \({\mathcal {N}}_{r,[r^1],n,\mu }\) with additional g hypermultiplets in the adjoint representation, all of them with twisted mass m, which reproduces an \({\mathcal {N}}=(0,2)^*\) theory.Footnote 1 In the same spirit as in [13], from the point of view of the matter fields this consists in introducing 2g adjoint chirals and 2g fundamental chirals, with appropriate relations dictated by the brane system. As it was the case for the theory without any adjoint hypermultiplet, each of the fundamentals we introduce makes the theory anomalous by breaking a chiral half of the supersymmetry, and this phenomenon can be cured by insertion of observables, encoded in \(\overline{\mathrm{D7}}\) contributions. The additional field content of Table 1 is summarized in Table 3, while the ADHM relation on the \(n_0\) node must be modified

$$\begin{aligned}{}[B_1^0,B_2^0]+\sum _{i=1}^g[B_3^{i\dag },B_4^{i\dag }]+IJ=0 \end{aligned}$$
(3.41)

and the relations (3.42)–(3.45) must be enforced through \(\chi _0^{(3,4),i}\), \(K_i\) and \(L_i\).

$$\begin{aligned} {\mathcal {E}}_{3,i}^{\mathrm{adj}}&=[B_1,B_3^i]-[B_2^\dag ,B_4^{i\dag }] \end{aligned}$$
(3.42)
$$\begin{aligned} {\mathcal {E}}_{4,i}^{\mathrm{adj}}&=[B_1,B_4^i]-[B_2^\dag ,B_3^{i\dag }]\end{aligned}$$
(3.43)
$$\begin{aligned} {\mathcal {E}}_{K_i}^{\mathrm{fun}}&=B_3^iI-B_4^{i\dag } J^\dag \end{aligned}$$
(3.44)
$$\begin{aligned} {\mathcal {E}}_{L_i^\dag }^{\mathrm{fun}}&=B_4^iI+B_3^{i\dag } J^\dag \end{aligned}$$
(3.45)
Table 3 Hypermultiplet additional fields for quiver in Fig. 13

The partition function for a general genus g Riemann surface \({\mathcal {C}}_g\) with one puncture will now read (we take the \(r=1\) case for the sake of simplicity)

$$\begin{aligned} {\mathcal {Z}}^{\mathrm{ell}}_1({\mathcal {C}}_g;q_0,\dots ,q_{s-1})=\sum _{\mu _1\subseteq \cdots \subseteq \mu _0}q_0^{|\mu _0|}q_1^{|\mu _0\setminus \mu _1|}\cdots q_{s-1}^{|\mu _0\setminus \mu _{s-1}|}Z_{(\mu _0,\mu _1,\dots ,\mu _{s-1})}^{\mathrm{ell},g}, \end{aligned}$$
(3.46)

with

$$\begin{aligned} Z_{(\mu _0,\mu _1,\dots ,\mu _{s-1})}^{\mathrm{ell},g}={\mathcal {L}}_{\mu _0}^{\mathrm{ell}}{\mathcal {N}}_{\mu _0}^{\mathrm{ell}}\overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}{\mathcal {E}}_{g,\mu _0}^{\mathrm{ell}}\overline{{\mathcal {E}}}_{g,\mu _0}^{\mathrm{ell}}{\mathcal {T}}_{\mu _0,\mu _1}^{\mathrm{ell}}\overline{{\mathcal {T}}}_{\mu _0,\mu _1}^{\mathrm{ell}}{\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}}^{\mathrm{ell}}, \end{aligned}$$
(3.47)

where we defined

$$\begin{aligned} {\mathcal {E}}_{g,\mu _0}^{\mathrm{ell}}&=\prod _{s\in Y_{\mu _0}}\theta _1^g(\tau |E(s)-m) \theta _1^g(\tau |E(s)-\epsilon +m), \end{aligned}$$
(3.48)
$$\begin{aligned} \overline{{\mathcal {E}}}_{g,\mu _0}^{\mathrm{ell}}&=\prod _{s\in \mu _0\setminus \Box }\frac{1}{\theta _1^g(\tau |\phi (s)-{\tilde{a}}-m) \theta _1^g(\tau |\phi (s)-{\tilde{a}}+\epsilon -m)}. \end{aligned}$$
(3.49)

We remark that by setting \(g=0\) we readily recover the function \(Z_{(\mu _0,\dots ,\mu _{s-1})}^{\mathrm{ell}}\) which is needed in order to compute the partition function \({\mathcal {Z}}_1^{\mathrm{ell}}(S^2;q_0,\dots ,q_{s-1})\).

In the same way as we did in Sect. 3.1.5, we can compute the full partition function \({\mathcal {Z}}_1^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\dots ,q_{s-1})\) by first summing over the nested partitions \(\mu _1\subseteq \cdots \subseteq \mu _{s-1}\) and use the definition (3.27) of \(P^{\mathrm{ell}}\) in order to get

$$\begin{aligned} {\mathcal {Z}}^{\mathrm{ell}}_1({\mathcal {C}}_g;q_0,\dots ,q_{s-1})=\sum _{\mu _0}q_0^{|\mu _0|} {\mathcal {Y}}^{\mathrm{ell}}_{g,\mu _0}P^{\mathrm{ell}}_{\mu _0}, \end{aligned}$$
(3.50)

with the following definition of \({\mathcal {Y}}^{\mathrm{ell}}_{g,\mu _0}\)

$$\begin{aligned} {\mathcal {Y}}^{\mathrm{ell}}_{g,\mu _0}={\mathcal {L}}_{\mu _0}^{\mathrm{ell}}{\mathcal {N}}_{\mu _0}^{\mathrm{ell}}\overline{{\mathcal {N}}}_{\mu _0}^{\mathrm{ell}}{\mathcal {E}}^{\mathrm{ell}}_{g,\mu _0}\overline{{\mathcal {E}}}^{\mathrm{ell}}_{g,\mu _0}. \end{aligned}$$
(3.51)

Again we remark that \(P_{\mu _0}^{\mathrm{ell}}\xrightarrow {q_{i>0}\rightarrow 0}1\) so that \({\mathcal {Z}}_{1}^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\dots ,q_{s-1})\xrightarrow {q_{i>0}\rightarrow 0}{\mathcal {Z}}_1^{\mathrm{ell}}({\mathcal {C}}_g;q_0)\).

3.3.1 Comet-shaped quiver

Finally, we are interested in computing the partition function on a Riemann surface \({\mathcal {C}}_g\) with k punctures of generic holonomy, whose low-energy GLSM is in general described by the quiver in Fig. 14.

Fig. 14
figure 14

Comet-shaped quiver

We will start from the case of \({\mathcal {C}}_0=S^2\), which will take the form (3.52)

$$\begin{aligned} \begin{aligned} {\mathcal {Z}}_k^{\mathrm{ell}}(S^2;q_0,\{q_1^i,\dots ,q_{s-1}^i\})&=\sum _{\mu _0}q_0^{|\mu _0|} \sum _{\{\mu _1^i\subseteq \cdots \subseteq \mu _{s-1}^i\}_{i=1}^k}\prod _{j=1}^k \left( q_1^{|\mu _0\setminus \mu _1^j|}\cdots \right. \\&\quad \left. \cdots q_{s-1}^{|\mu _0\setminus \mu _{s-1}^j|}\right) Z_{(\mu _0,\{\mu ^i_1,\dots , \mu ^i_{s-1}\})}^{\mathrm{ell}}. \end{aligned}\nonumber \\ \end{aligned}$$
(3.52)

In this case the virtual tangent space to \({\mathcal {N}}_{r,[r^1],n,\mu }\) in (3.15) will be modified to be of the form (3.53).

$$\begin{aligned} \begin{aligned} T_Z^{\text {vir}}{\mathcal {N}}(r,\{n_0^i,\ldots ,n_{s-1}^i\})&={{\,\mathrm{End}\,}}(V_0)\otimes (Q-1-\Lambda ^2Q)+{{\,\mathrm{Hom}\,}}(W,V_0)\\&\quad +{{\,\mathrm{Hom}\,}}(V_0,W)\otimes \Lambda ^2Q-\sum _{i=1}^k{{\,\mathrm{Hom}\,}}(V_1^{(k)},W)\otimes \Lambda ^2Q\\&\quad +\sum _{i=1}^k\sum _{\ell =1}^{s-1}\bigg [\left( {{\,\mathrm{End}\,}}(V_{\ell }^{(k)}) -{{\,\mathrm{Hom}\,}}(V_{\ell }^{(k)},V_{\ell -1}^{(k)})\right) \otimes \\&\quad \otimes (Q-1-\Lambda ^2Q)\bigg ]. \end{aligned}\nonumber \\ \end{aligned}$$
(3.53)

By a simple generalization of the computations leading to (3.16) it is possible to see that \({\mathcal {Z}}_k^{\mathrm{ell}}(S^2;q_0,\{q_1^i,\dots ,q_{s-1}^i\})\) takes a form similar to (3.28), as is shown in (3.54)

$$\begin{aligned} {\mathcal {Z}}_k^{\mathrm{ell}}(S^2;q_0,\{q_1^i,\dots ,q_{s-1}^i\})=\sum _{\mu _0}q_0^{|\mu _0|}{\mathcal {Y}}^{\mathrm{ell}}_{\mu _0}\prod _{i=1}^kP^{\mathrm{ell},i}_{\mu _0}, \end{aligned}$$
(3.54)

with

$$\begin{aligned} P^{\mathrm{ell},i}_{\mu _0}=\sum _{\mu _1^i\subseteq \cdots \subseteq \mu _{s-1}^i} {\mathcal {T}}_{\mu _0,\mu _1^i}^{\mathrm{ell}}\overline{{\mathcal {T}}}_{\mu _0,\mu _1^i}^{\mathrm{ell}}{\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}^i}^{\mathrm{ell}}\left( q_1^i\right) ^{|\mu _0\setminus \mu _1^i|}\cdots \left( q_{s-1}^i\right) ^{|\mu _0\setminus \mu _{s-1}^i|},\nonumber \\ \end{aligned}$$
(3.55)

and the functions \({\mathcal {T}}_{\mu _0,\mu _1^i}^{\mathrm{ell}}\), \(\overline{{\mathcal {T}}}_{\mu _0,\mu _1^i}^{\mathrm{ell}}\) and \({\mathcal {W}}_{\mu _0,\dots ,\mu _{s-1}^i}\) take the same form as in Eqs. (3.19)–(3.24).

By a completely analogous procedure we can get that partition function of the low-energy theory relative to a general Riemann surface of genus g, possibly \(g=0\). By using the results of Sect. 3.3, we easily see that

$$\begin{aligned} \begin{aligned} {\mathcal {Z}}_k^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\{q_1^i,\dots ,q_{s-1}^i\})&=\sum _{\mu _0}q_0^{|\mu _0|}\sum _{\{\mu _1^i\subseteq \cdots \subseteq \mu _{s-1}^i\}_{i=1}^k}\prod _{j=1}^k\left( q_1^{|\mu _0\setminus \mu _1^j|}\cdots \right. \\&\quad \left. \cdots q_{s-1}^{|\mu _0\setminus \mu _{s-1}^j|}\right) Z_{(\mu _0,\{\mu ^i_1,\dots , \mu ^i_{s-1}\})}^{\mathrm{ell},g}. \end{aligned} \end{aligned}$$
(3.56)

By turning on the matter bundle described in Sect. 3.3 on the moduli space of nested instantons \({\mathcal {N}}(r,n_0,\{n_1^i,\dots ,n_{s-1}^i\})\), whose virtual tangent space is given in Eq. (3.53) as an element of the representation ring of the torus R(T), the suspersymmetric localization theorem (or equivalently the equivariant one) gives us (3.57),

$$\begin{aligned} {\mathcal {Z}}_k^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\{q_1^i,\dots ,q_{s-1}^i\})=\sum _{\mu _0}q_0^{|\mu _0|} {\mathcal {Y}}^{\mathrm{ell}}_{g,\mu _0}\prod _{i=1}^kP^{\mathrm{ell},i}_{\mu _0}, \end{aligned}$$
(3.57)

where \(P^{\mathrm{ell},i}_{\mu _0}\) is defined in (3.55) and \({\mathcal {Y}}_{g,\mu _0}^{\mathrm{ell}}\) is the same one as in Eq. (3.51).

A couple of final remarks are due here. First of all we notice that we can switch off any number of the contributions of the tails of the comet-shaped quiver in Fig. 14 by taking the limit to 0 of the respective instanton counting parameters. Then, given any \(k'<k\) we have that

$$\begin{aligned} {\mathcal {Z}}_k^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\{q_1^i,\dots ,q_{s-1}^i\}) \xrightarrow [\begin{array}{c} i=1,\dots ,s-1\\ j=k'+1,\dots ,k \end{array}]{q_i^j\rightarrow 0}{\mathcal {Z}}_{k'}^{\mathrm{ell}}({\mathcal {C}}_g;q_0,\{q_1^i,\dots ,q_{s-1}^i\}). \end{aligned}$$
(3.58)

Moreover, we expect our partitions functions to be computing the equivariant elliptic cohomology of the moduli spaces of stable representations of quivers in Figs. 10, 11, 12 and 14, as in [67].

3.4 Limit to supersymmetric quantum mechanics

We now want to study a particular dimensional reduction of the 2d \({\mathcal {N}}=(0,2)\) system we studied on \(T^2\) in the previous subsections. By reducing on a circle we get the Witten index of an \({\mathcal {N}}=2\) SQM. This dimensional reduction can be obtained from the elliptic case we just studied by taking the limit \({\text{ e }}^{2\pi {\text{ i }}\tau }\rightarrow 0\). In this scaling limit we can use the fact that \(\theta _1(\tau |z)\rightarrow 2q^{1/8}\sin (\pi z)\) as \(q={\text{ e }}^{2\pi {\text{ i }}\tau }\rightarrow 0\). In the resulting theory on \(S^1\) we can decouple the \(\overline{\mathrm{D7}}\) branes by taking very large values of the Cartan parameter \({\overline{a}}\) and then rescaling the gauge coupling. Moreover, the quantum mechanical partition function can be obtained by itself via localization, and the result agrees with the decoupling procedure we just described. Indeed, the \({\overline{{\mathrm{D7}}}}\)-branes only act as a source of observables matching the anomaly, so they do not give rise to new poles in the localization integral. Moreover, the observables they generate do not contribute in the dimensionally reduced theory, whose moduli space is zero-dimensional, i.e. it does not display unbalanced fermionic zero modes. As we already anticipated, we will see how the results we obtain by this procedure compute particular equivariant virtual invariants of the bundle \({\mathcal {V}}_{g}\) over the moduli space of nested instantons \({\mathcal {N}}_{r,[r^1],n,\mu }\), which is described by the stable representations of the quiver in Fig. 13. A bit of care is required in order to take the correct scaling limit, and in particular one has to require that \(q\rightarrow 0\), while \({{\,\mathrm{vol}\,}}(T^2)\rightarrow \beta =r_{S^1}\). Moreover, one should take into account that in the \(S^1\) theory twisted masses are also rescaled by \(\beta \), so that the result may be expressed in terms of \(q_1={\text{ e }}^{\beta \epsilon _1/2}\), \(q_2={\text{ e }}^{\beta \epsilon _2/2}\) and \(y={\text{ e }}^{-\beta m}\).

The geometric interpretation of the Witten index of the quiver gauge theories described in the previous section is the equivariant (virtual) Euler characteristic of a given bundle over the moduli space of nested instantons. Then, computing the Witten index geometrically amounts to studying the stable representations in the category of vector spaces of the quiver in Fig. 4 under suitable stability conditions. This procedure has the advantage of letting us compute the weight decomposition of the virtual tangent space \(T^{\mathrm{vir}}_Z{\mathcal {N}}_{r,[r^1],n,\mu }\) at the fixed points Z in the representation ring of the torus. The way in which this is done is very briefly described in Sect. 3.1.3. As it is shown in Sect. 3.1.3, the fixed locus of the torus action consists only of isolated points, which are characterized in terms of s-tuples of nested colored partitions \(\varvec{\mu }_1\subseteq \cdots \varvec{\mu }_{s-1}\subseteq \varvec{\mu }_0\), such that \(|\varvec{\mu }_0|=n_0=n\), while \(|\varvec{\mu }_j|=n_0-n_j\).

Once the fixed point locus has been completely characterized and a weight decomposition of the virtual tangent space is at hand, one can in full generality define an s-parameter family of partition functions on \({\mathcal {N}}_{r,[r^1],n,\mu }\), with parameters \(\mathbf{p }=(p_0,p_1,\dots ,p_{s-1})\in {\mathbb {Z}}^{s}\). In terms of the quiver vector spaces \((W,V_0,\dots ,V_{s-1})\) one can introduce \((s+1)\)-tautological bundles \({\mathcal {W}}\) and \({\mathcal {V}}_i\), \(i=0,\dots ,s-1\), with \({\mathcal {W}}={\mathcal {O}}_{{\mathcal {N}}_{r,[r^1],n,\mu }}\). We can then define \({\mathcal {L}}_i=\det {\mathcal {V}}_i\), \({\mathcal {L}}_{\mathbf{p }}=\bigotimes _i{\mathcal {L}}_i^{\otimes p_i}\) and compute the virtual Euler characteristic of the bundle \(S\otimes {\mathcal {L}}_{\mathbf{p }}\) over \({\mathcal {N}}_{r,[r^1],n,\mu }\), with S an arbitrary irreducible representation of T. The generating function of the virtual Euler characteristics of the moduli space of nested instantons in (3.59) will then reproduce the QM partition function 2.21, when \(\mathbf{p }=(1-g,0,\dots ,0)\).Footnote 2

$$\begin{aligned} Z_{\mathbf{p }}^{\text {vir}}(q_1,q_2,\mathbf{x })=\sum _{\mathbf{n } \in {\mathbb {Z}}^{s}_{\ge 0}}{{\,\mathrm{ch}\,}}_{T}\chi ^{\text {vir}}_{T}\left( {\mathcal {N}}_{r,[r^1],n,\mu }, {\mathcal {L}}_{\mathbf{p }}\right) \prod _{i=1}^{s}x_i^{n_i}. \end{aligned}$$
(3.59)

In the following we use the notation \({{\,\mathrm{ch}\,}}_T\) to denote the T-equivariant Chern character of a vector bundle, which has a very convenient representation in the representation ring R(T). The usual Chern character is defined as follows: If E is rank r vector bundle over X, with Chern roots \(x_1,\dots ,x_r\), then one defines

$$\begin{aligned} {{\,\mathrm{ch}\,}}(E)=\sum _{i=1}^r{\text{ e }}^{x_i}, \end{aligned}$$
(3.60)

which can be equivariantly extended to a ring homomorphism \({{\,\mathrm{ch}\,}}_G:K^i_G(X)\rightarrow H^i_G({\hat{X}},{\mathbb {C}})\), where \({\hat{X}}=\{(x,g)\in X\times G|xg=x\}=\coprod _gX^g\) and \(H^i_G({\hat{X}},{\mathbb {C}})\simeq \left[ \bigoplus _gH^i(X^g,{\mathbb {C}})\right] ^G\). The effect of \({{\,\mathrm{ch}\,}}_G\) can be concretely characterized as follows: If E is a G-equivariant vector bundle on X, for each \(x\in X^g\), we can compute the eigenvalues (supposed to be distinct) \(\lambda _1,\dots ,\lambda _r\) of the G-action, and the corresponding eigenspaces \(E_x^1,\dots ,E_x^r\), so that \(E|_{X^g}\) can be represented as the direct sum of vector bundles

$$\begin{aligned} E_{X^g}=E^1\oplus \cdots \oplus E^r. \end{aligned}$$
(3.61)

Finally, one defines \({{\,\mathrm{ch}\,}}_g(E)=\sum _i\lambda _i{{\,\mathrm{ch}\,}}(E^i)\), so that

$$\begin{aligned} {{\,\mathrm{ch}\,}}_G(E)=\bigoplus _{g\in G}{{\,\mathrm{ch}\,}}_g(E)\in \left[ \bigoplus _{g\in G}H^{\mathrm{ev}}(X^g,{\mathbb {C}})\right] ^G. \end{aligned}$$
(3.62)

The Chern character, and also the equivariant Chern character, satisfies some important properties which we will use extensively in the following:

$$\begin{aligned} {{\,\mathrm{ch}\,}}(E\oplus F)={{\,\mathrm{ch}\,}}E+{{\,\mathrm{ch}\,}}F,\qquad {{\,\mathrm{ch}\,}}(E\otimes F)={{\,\mathrm{ch}\,}}E{{\,\mathrm{ch}\,}}F. \end{aligned}$$
(3.63)

If we restrict to the case \(\mathbf{p }=(p_0,0,\dots ,0)\), the fiber of \({\mathcal {L}}_{\mathbf{p }}\) at a fixed point \(Z\leftrightarrow \varvec{\mu }_1\subseteq \cdots \subseteq \varvec{\mu }_{s-1} \subseteq \varvec{\mu }_0\) will be given by (3.64),

$$\begin{aligned} {\mathcal {L}}_Z={\mathcal {L}}_{\varvec{\mu }_0}=\left( \prod _{\alpha =1}^r \prod _{i=1}^{M_0^{(a)}}\prod _{j=1}^{\mu _{0,i}^{(a)\prime }}T_{a_\alpha }T_1^{-i+1} T_2^{-j+1}\right) ^{p_0}, \end{aligned}$$
(3.64)

where \((T_1,T_2,T_{a_1},\dots ,T_{a_r})\) denote the fundamental characters of \(T\times \left( {\mathbb {C}}^*\right) ^r\) acting on \({\mathcal {N}}_{r,[r^1],n,\mu }\) and \(M_0^{(a)}=\mu _{0,1}^{(a)\prime }\), \(N_0^{(a)}=\mu _{0,1}^{(a)}\).

Then, supersymmetric localization (equivalently equivariant localization) can be exploited in order to compute partition functions (equivalently virtual equivariant Euler characteristics). If we start from the case \(g=0\) we get

$$\begin{aligned} \begin{aligned} {{\,\mathrm{ch}\,}}_{T}\left[ \chi _{T}^{\text {vir}}\left( {\mathcal {N}}(r,\mathbf{n }), {\mathcal {L}}_{\mathbf{p }}\right) \right]&=\sum _{Z\in {\mathcal {N}}_{r,[r^1],n, \mu }^{T}}\frac{{{\,\mathrm{ch}\,}}_{T}{\mathcal {L}}_Z}{\Lambda _{-1}\left[ T_Z^{\text {vir}} {\mathcal {N}}_{r,[r^1],n,\mu }^\vee \right] }\\&=\sum _{\varvec{\mu }_1\subseteq \cdots \subseteq \varvec{\mu }_{s-1}\subseteq \varvec{\mu }_0}\left( \frac{{\mathcal {L}}_{\varvec{\mu }_0}(q_1,q_2)}{\Lambda _{-1}\left[ T_{{\tilde{Z}}}{\mathcal {M}}_{r,n_0}^\vee \right] } {\mathcal {T}}_{\varvec{\mu }_0,\varvec{\mu }_1}(q_1,q_2)\cdot \right. \\&\quad \cdot {\mathcal {W}}_{\varvec{\mu }_{0},\dots ,\varvec{\mu }_{s-1}}(q_1,q_2)\Bigg ), \end{aligned}\nonumber \\ \end{aligned}$$
(3.65)

with \(\Lambda _t(E)=\sum _{i\ge 0}t^i\Lambda ^i E\) for any (equivariant) vector bundle E on \({\mathcal {N}}_{r,[r^1],n,\mu }\), while \({\mathcal {L}}_{\varvec{\mu }_0}(q_1,q_2)\), \({\mathcal {W}}_{\varvec{\mu }_0,\dots ,\varvec{\mu }_{s-1}}(q_1,q_2)\) and \({\mathcal {T}}_{\varvec{\mu }_{0},\varvec{\mu }_1}(q_1,q_2)\) are given by Eqs. (3.66)–(3.68),

$$\begin{aligned} {\mathcal {L}}_{\varvec{\mu }_0}(q_1,q_2)&=\left( \prod _{a=1}^r\prod _{i=1}^{M_0^{(a)}} \prod _{j=1}^{\mu _{0,i}^{(a)\prime }}\rho _a q_1^{i-1}q_2^{j-1}\right) ^{p_0} \end{aligned}$$
(3.66)
$$\begin{aligned} {\mathcal {T}}_{\varvec{\mu }_0,\varvec{\mu }_1}(q_1,q_2)&=\prod _{a=1}^r \prod _{i=1}^{M_0^{(a)}}\prod _{j=1}^{\mu _{0,i}^{(a)}-\mu _{1,i}^{(a)}}\left( 1-\rho _a q_1^{-i}q_2^{-j-\mu _{1,i}^{(a)\prime }}\right) \end{aligned}$$
(3.67)
$$\begin{aligned} {\mathcal {W}}_{\varvec{\mu }_{0},\dots ,\varvec{\mu }_{s-1}}(q_1,q_2)&= \prod _{k=0}^{s-2}\prod _{a,b=1}^r\prod _{i=1}^{M_0^{(a)}}\prod _{j=1}^{N_0^{(b)}} \left( \frac{(1-\rho _a\rho _b^{-1}q_1^{\mu _{k,j}^{(b)}-i}q_2^{j-\mu _{k+1,i}^{(a) \prime }-1})}{(1-\rho _a\rho _b^{-1}q_1^{\mu _{k+1,j}^{(b)}-i}q_2^{j-\mu _{k+1,i}^{(a) \prime }-1})}\right. \nonumber \\&\quad \left. \frac{(1-\rho _a \rho _b^{-1}q_1^{\mu _{k+1,j}^{(b)} -i}q_2^{j-\mu _{0,i}^{(a)\prime }-1})}{(1-\rho _a\rho _b^{-1}q_1^{\mu _{k,j}^{(b)} -i}q_2^{j-\mu _{0,i}^{(a)\prime }-1})}\right) \end{aligned}$$
(3.68)

with \(\rho _i={{\,\mathrm{ch}\,}}T_{a_i}\) and similarly \(q_i={{\,\mathrm{ch}\,}}T_i\).

The generalization to the case of a general Riemann surface \({\mathcal {C}}_g\) of genus g is immediate, as it only amounts to computing the “virtual Hirzebruch \(\chi _y\)-genus” of the bundle \(\pi ^*{\mathcal {V}}_g\rightarrow {\mathcal {N}}_{r,[r^1],n,\mu }\). This is obviously the same as turning on a matter bundle relative to additional g adjoint hypermultiplets, whose twisted mass m is naturally identified with y in the Hirzebruch genus by exponentiation.

$$\begin{aligned} \begin{aligned} {{\,\mathrm{ch}\,}}_{T}\chi _y^{T,\mathrm{vir}}\left( \pi ^*{\mathcal {V}}_g,{\mathcal {N}}_{r,[r^1],n, \mu }\right)&=\sum _{\varvec{\mu }_1\subseteq \cdots \subseteq \varvec{\mu }_{s-1} \subseteq \varvec{\mu }_0}\left( \frac{{{\,\mathrm{ch}\,}}_{T}({\mathcal {L}}_{\varvec{\mu }_0}) {{\,\mathrm{ch}\,}}_{T}\Lambda _{y}[(T_{\varvec{\mu }_0}{\mathcal {M}}_{r,n_0}^\vee )^{\oplus g}]}{{{\,\mathrm{ch}\,}}_{T}\Lambda _{-1}[T_{\varvec{\mu }_0}{\mathcal {M}}_{r,n_0}^\vee ]}\cdot \right. \\&\quad \left. \cdot {\mathcal {T}}_{\varvec{\mu }_1,\varvec{\mu }_0}(q_1,q_2) \prod _{i=0}^{s-2}{\mathcal {W}}_{\varvec{\mu }_{i+1},\varvec{\mu }_i}^{\varvec{\mu }_0}(q_1,q_2)\right) . \end{aligned} \end{aligned}$$
(3.69)

Surprisingly enough, explicit computations suggest that the partition function of each choice of numerical type for the nested instantons quiver should consists of a usual Nekrasov partition function multiplied by a polynomial in the torus characters. This observation is summarized in the following conjecture.

Conjecture 1 The function \(\sum _{\varvec{\mu }_{i>0}}{\mathcal {T}}_{\varvec{\mu }_0,\varvec{\mu }_1} (q_1,q_2){\mathcal {W}}_{\varvec{\mu }_0,\dots ,\varvec{\mu }_{s-1}}(q_1,q_2)\) is a polynomial in \(q=q_1^{-1}\) and \(t=q_2^{-1}\) with rational coefficients in the \(\{\rho _i\}_{1\le i\le r}\), while it is a polynomial with integer coefficients when \(r=1\).

3.5 Comparison to LHRV formulae

The Nekrasov partition function on \({\mathbb {R}}^4\times S^1\) is known to have the following form

$$\begin{aligned} Z_{k,N}^{{\mathbb {R}}^4\times S^1}=\sum _{\mathbf{Y }_k}\prod _{\lambda ,{\tilde{\lambda }}=1}^N\prod _{s\in Y_\lambda }\frac{\sinh \left[ \frac{\beta }{2}(E(s)-m))\right] \sinh \left[ \frac{\beta }{2}(E(s)-\epsilon +m)\right] }{\sinh \left[ \frac{\beta }{2}E(s)\right] \sinh \left[ \frac{\beta }{2}(E(s)-\epsilon )\right] },\qquad \end{aligned}$$
(3.70)

where \(E(s)=a_{\lambda {\tilde{\lambda }}}-\epsilon _1h(s)+\epsilon _2(v(s)+1)\), and given two Young diagrams \(Y_\lambda ,Y_{{\tilde{\lambda }}}\in \mathbf{Y }_k\) the quantities h(s) and v(s) are defined to be \(h(s)=\nu _{i_\lambda }-j_\lambda \) and \(v(s)={\tilde{\nu }}'_{j_\lambda }-i_\lambda \). We will be interested in the specialization of the Nekrasov partition function to the case \(N=1\), so that h(s) and v(s) will become, respectively, the arm length a(s) and leg length l(s) for the box s in the Young tableaux classifying a given pole configuration. Now, following the conventions of [55], let \(\mathbf{x }_1=\{x_{1,1},x_{1,2},\dots \}\) and \(\mathbf{x }_k=\{x_{k,1},x_{k,2},\dots \}\) be k infinite sets of variables and let moreover \(\Lambda (\mathbf{x }_1),\dots ,\Lambda (\mathbf{x }_k)\) be the corresponding rings of symmetric functions. Given a partition \(\lambda \), \({\tilde{H}}_\lambda (\mathbf{x };q,t)\in \Lambda (\mathbf{x }) \otimes _{{\mathbb {Z}}}{\mathbb {Q}}[q,t]\) will denote the modified Macdonald symmetric function. The k-point genus g Cauchy function \(\Omega (z,w)\), with coefficients in \({\mathbb {Q}}[z,w]\otimes _{{\mathbb {Z}}}\Lambda (\mathbf{x }_1,\dots ,\mathbf{x }_k)\), is defined as follows:

$$\begin{aligned} \Omega (z,w)=\sum _{\lambda \in {\mathcal {P}}}{\mathcal {H}}_\lambda (z,w) \prod _{i=1}^k{\tilde{H}}_\lambda (\mathbf{x }_i;z^2,w^2), \end{aligned}$$
(3.71)

with

$$\begin{aligned} {\mathcal {H}}_\lambda (z,w)=\prod _{s\in \lambda }\frac{(z^{2a(s)+1}-w^{2l(s)+1})^{2g}}{(z^{2a(s)+2-w^{2l(s)}})(z^{2a(s)}-w^{2l(s)+2})}. \end{aligned}$$
(3.72)

The modified Macdonald polynomials \({\tilde{H}}_\lambda (\mathbf{x };q,t)\) are defined as

$$\begin{aligned} {\tilde{H}}_\lambda (\mathbf{x };q,t)=\sum _{\mu }{\tilde{K}}_{\mu \lambda } (q,t)s_\mu (\mathbf{x }), \end{aligned}$$
(3.73)

where \(s_\lambda (\mathbf{x })\) are the usual Schur functions, while \({\tilde{K}}_{\lambda \mu }(q,t)\) denotes the modified Kostka polynomials, which are expressed in terms of the usual Kostka polynomials as

$$\begin{aligned} {\tilde{K}}_{\lambda \mu }(q,t)=t^{n(\mu )}K_{\lambda \mu }(q,t^{-1}), \end{aligned}$$
(3.74)

with \(n(\mu )=\sum _{i=1}^{l(\mu )}\mu _i(i-1)\), and \(K_{\lambda \mu }(q,t)\) can be interpreted as being a deformation of the Kostka coefficients \(K_{\lambda \mu }\) appearing in the expansion of the Schur polynomials in terms of the monomial symmetric functions:

$$\begin{aligned} s_\lambda (\mathbf{x })=\sum _{\mu }K_{\lambda \mu }m_\mu (\mathbf{x }). \end{aligned}$$
(3.75)

Moreover, the modified Macdonald polynomials can be viewed as a q-deformation of the standard Hall–Littlewood polynomials, and are related in a non-trivial way to the Macdonald polynomials \(P_\mu (\mathbf{x };q,t)\), which are eigenfunctions of the trigonometric Ruijsenaars–Schneider Hamiltonian [70, 71]:

$$\begin{aligned} {\tilde{H}}_\lambda [X;q,t]=t^{n(\lambda )}J_\lambda \left[ \frac{X}{1-1/t};q,1/t\right] , \end{aligned}$$
(3.76)

where X denotes the plethystic substitution \(X=x_1+x_2+x_3+\cdots \), the square brackets are to be intended as a plethystic insertion and

$$\begin{aligned} J_\lambda (\mathbf{x };q,t)=\prod _{s\in \lambda }\left( 1-q^{a_\lambda (s)}t^{l_\lambda (s) +1}\right) P_\lambda (\mathbf{x };q,t). \end{aligned}$$
(3.77)

The modified Macdonald polynomials are also eigenfunctions of a linear operator \(\Delta \), [72], which acts on a symmetric function f as

$$\begin{aligned} \Delta f=\left. f\left[ X+\frac{(1-q)(1-t)}{z}\right] \Omega [-zX]\right| _{z^0}, \end{aligned}$$
(3.78)

where \(\Omega [X]=\sum _{n=0}^\infty h_n(X)\) and \((\bullet )|_{z^0}\) denotes the constant part in z.

We will think to \(\Omega (z,w)\) as being a function associated with a genus g Riemann surface with k punctures. Moreover, if we are give \(\varvec{\mu }=(\mu ^1,\dots ,\mu ^k)\in {\mathcal {P}}^k\) we can define the following function

$$\begin{aligned} {\mathbb {H}}_{\varvec{\mu }}(z,w)=(z^2-1)(1-w^2)\langle {{\,\mathrm{PL}\,}}\Omega (z,w), h_{\varvec{\mu }}\rangle , \end{aligned}$$
(3.79)

where \(h_{\varvec{\mu }}=h_{\mu ^1}(\mathbf{x }_1)\cdots h_{\mu ^k}(\mathbf{x }_k)\in \Lambda (\mathbf{x }_1,\dots ,\mathbf{x }_k)\) are the complete symmetric functions, and \(\langle \cdot ,\cdot \rangle \) is an extension of the Hall pairing. The interest in \({\mathbb {H}}_{\varvec{\mu }}(z,w)\) lays in the fact that it encodes information both about \(GL_n({\mathbb {C}})\) character varieties \({\mathcal {M}}_{\varvec{\mu }}\) of k-punctured genus g Riemann surfaces with generic semisimple conjugacy classes of type \(\varvec{\mu }\) at the punctures and about comet-shaped quivers \({\mathcal {Q}}_{\varvec{\mu }}\) with g loops and k tails of length defined by \(\varvec{\mu }\). It is in fact conjectured that through the knowledge of \({\mathbb {H}}_{\varvec{\mu }}(z,w)\) we can get the mixed Hodge polynomial and the E-polynomial (and thus the Euler characteristic) of both these character varieties and quiver varieties.

If we now study the particular case of comet-shaped quivers with \(k=1\), \(l(\mu )=1\) and \(g=1\), whose corresponding quiver is the Jordan quiver, we can specialize \(\mathbf{x }=(T,0,\dots )\) for some variable T and \({\tilde{H}}_\lambda (T,0,\dots ;z,w)=T^{|\lambda |}\), so that

$$\begin{aligned} \Omega (z,w)=\sum _{k}\sum _{|\lambda |=k}\prod \frac{\left( z^{2a(s)+1}-w^{2l(s) +1}\right) ^2}{\left( z^{2a(s)+2}-w^{2l(s)}\right) \left( z^{2a(s)} -w^{2l(s)+2}\right) }T^{|\lambda |}. \end{aligned}$$
(3.80)

If we now compare (3.80) to (3.70) in the case \(N=1\), with \(m=\epsilon /2\), we can immediately see how closely \(\Omega (z,w)\) resembles to \(\sum _{k}Z_{k,1}^{{\mathbb {R}}^4\times S^1}q^k\) as long as we make the identifications \(z^2={\text{ e }}^{\beta \epsilon _1}\), \(w^2={\text{ e }}^{\beta \epsilon _2}\) and \(T=q\), q being the instanton counting parameter.

If we next take g to be arbitrary, but still take \(k=1\) and \(l(\mu )=1\) a generalization of our previous observations is straightforward. In fact, as we already pointed out in the previous sections, adding loops to the Jordan quiver has the net effect of introducing \(2g+2\) matter fields \(B_1\), \(B_2\), \(B_3^{(i)}\), \(B_4^{(i)}\) (with \(i=1,\dots ,g\)) transforming in the adjoint representation of the gauge group U(k). The role played by each of the \(B_3^{(i)}\), \(B_4^{(i)}\) fields is analogous to the one of \(B_3\) and \(B_4\) in the ADHM linear sigma model with adjoint matter. Since all of these fields do not contribute with poles to the residue computation of the localization formula, if we choose their twisted masses and R-charges to be the same as the ones for \(B_3\) and \(B_4\) their net effect will be that of introducing a gth power to the numerator of (3.70) (which really is the meaning of turning on a matter bundle for g adjoint hypermultiplets twisted by their mass m).

Actually one needs to turn on a Chern–Simons coupling in order to exactly reproduce \(\Omega (z,w)\) starting from a gauge theory. In fact we can rewrite (3.80) as

$$\begin{aligned} \begin{aligned} \Omega (z,w)=&\sum _k\sum _{|\lambda |=k}\prod _{s\in \lambda } \left[ (-1)^{g-1}\frac{(z^{2a(s)+1}w^{2l(s)+1})^g}{z^{2a(s)+2}w^{2l(s)+2}}\cdot \right. \\&\left. \cdot \frac{(1-z^{-2a(s)-1}w^{2l(s)+1})^g(1-z^{2a(s)+1}w^{-2l(s)+1})^g}{(1-z^{-2a(s)-2}w^{2l(s)})(1-z^{2a(s)}w^{-2l(s)-2})}T^{|\lambda |}\right] \end{aligned} \end{aligned}$$
(3.81)

and we can easily see that

$$\begin{aligned} \begin{aligned} \prod _{s\in \lambda }\frac{(z^{2a(s)+1}w^{2l(s)+1})^g}{z^{2a(s)+2}w^{2l(s)+2}}&=\frac{1}{(zw)^{|\lambda |}}\prod _{s\in \lambda }(z^{2a(s)+2}w^{2l(s)+2})^{g-1}\\&=\frac{1}{(zw)^{|\lambda |}}\left( z^{2\sum _s(a(s)+1)}w^{2\sum _s(l(s)+1)} \right) ^{g-1}\\&=\frac{1}{(zw)^{|\lambda |}}\left( z^{2\sum _s i(s)}w^{2\sum _s j(s)}\right) ^{g-1}\\&=\frac{(zw)^{|\lambda |(2g-2)}}{{\text{ e }}^{a(g-1)|\lambda |}(zw)^{|\lambda |}} \prod _{s\in \lambda }\left( {\text{ e }}^az^{2(i(s)-1)}w^{2(j(s)-1)}\right) ^{g-1}, \end{aligned} \end{aligned}$$

which apart from a harmless overall normalization, is the contribution of a Chern–Simons interaction at level \(1-g\), [56]. Thus, we conclude that the partition function for the 5d \({\mathcal {N}}=1^*\) ADHM quiver theory with g adjoint hypermultiplets and a Chern–Simons term at level \(1-g\) reproduces the Cauchy function (3.82) when resummed over all the instanton sectors (see also [73]).Footnote 3

$$\begin{aligned} \Omega (z,w)=\sum _{k}\sum _{|\lambda |=k}\prod \frac{\left( z^{2a(s)+1} -w^{2l(s)+1}\right) ^{2g}}{\left( z^{2a(s)+2}-w^{2l(s)}\right) \left( z^{2a(s)}-w^{2l(s)+2}\right) }T^{|\lambda |}. \end{aligned}$$
(3.82)

As it was shown in [21, 48], one interesting thing to point out in Eq. (3.82) is that it computes a generating function for a geometric index. It is actually known that the moduli space of stable representations for the ADHM data (3.83) is isomorphic to the Hilbert scheme of \(\dim (V)=n\) points in \({\mathbb {C}}^2\) when \(\dim (W)=1\).

figure a

Then, \(\Omega _\lambda (q_1,q_2)\) such that \(\Omega (z,w)=\sum _k\Omega _\lambda (z^2,w^2)T^{|\lambda |}\) is computing the Hirzebruch \(\chi _y\)-genus of a vector bundle over \(({\mathbb {C}}^2)^{[n]}\). In particular we have [21, 74]

$$\begin{aligned} \begin{aligned} \sum _{\lambda \in {\mathcal {P}}(n)}\Omega _\lambda (q_1,q_2,y)&={{\,\mathrm{ch}\,}}_{ T}\chi _y \left[ \left( T^\vee ({\mathbb {C}}^2)^{[n]}\right) ^{\oplus g}\otimes \left( \det {\mathcal {T}}\right) ^{1-g},({\mathbb {C}}^2)^{[n]}\right] \\&=\sum _{\lambda \in {\mathcal {P}}(n)}\frac{{{\,\mathrm{ch}\,}}_{ T}\left( \det {\mathcal {T}}\right) ^{1-g}{{\,\mathrm{ch}\,}}_{ T}\Lambda _{y}\left[ (T_\lambda ^\vee ({\mathbb {C}}^2)^{[n]})^{\oplus g}\right] }{{{\,\mathrm{ch}\,}}_{ T}\Lambda _{-1}\left[ T^\vee _\lambda ({\mathbb {C}}^2)^{[n]}\right] }, \end{aligned}\nonumber \\ \end{aligned}$$
(3.84)

where \(\det {\mathcal {T}}\) denotes the determinant line bundle on \(({\mathbb {C}}^2)^{[n]}\) and \(y={\text{ e }}^{-m}\).

It was proved in [21] that a similar result holds true also for the genus g Cauchy function relative to punctured Riemann surfaces with non-trivial holonomy around the punctures. In the case of a single puncture (assumed to be generic) of type \(\mu \), the Cauchy function at fixed \(|\lambda |=n\) computes the residual equivariant Hirzebruch genus of a vector bundle over a nested Hilbert scheme of n points \({\mathcal {N}}_{1,[1^1],n,\varvec{\mu }}\) on \({\mathbb {C}}^2\):

$$\begin{aligned} \sum _{\lambda \in {\mathcal {P}}(n)}{\mathcal {H}}_\lambda (z,w){\tilde{H}}_\lambda (\mathbf{x };z^2,w^2)={{\,\mathrm{ch}\,}}_{ T}\chi _y\left[ \pi ^*{\mathcal {V}}_{g},{\mathcal {N}}_{1,[1^1],n,\varvec{\mu }}\right] , \end{aligned}$$
(3.85)

where \(\pi :{\mathcal {N}}_{1,[1^1],n,\varvec{\mu }}\rightarrow ({\mathbb {C}}^2)^{[n]}\) is the natural projection of the nested Hilbert scheme to the underlying Hilbert scheme of n points on \({\mathbb {C}}^2\), and \({\mathcal {V}}_g=\left( T^\vee ({\mathbb {C}}^2)^{[n]}\right) ^{\oplus g}\otimes (\det {\mathcal {T}})^{1-g}\). Moreover, the rhs of (3.85) can be computed in terms only of characters of vector bundles over \(({\mathbb {C}}^2)^{[n]}={{\,\mathrm{Hilb}\,}}^n({\mathbb {C}}^2)\) due to a result by Haiman, [21, 75], and we have that

$$\begin{aligned}&{{\,\mathrm{ch}\,}}_{T}\chi _y\left[ \pi ^*{\mathcal {V}}_{g},{\mathcal {N}}_{1,[1^1],n, \varvec{\mu }}\right] \nonumber \\&=\sum _{\lambda \in {\mathcal {P}}(n)}\frac{{{\,\mathrm{ch}\,}}_{ T}\left( \det {\mathcal {T}}\right) ^{1-g}{{\,\mathrm{ch}\,}}_{ T}\Lambda _{y}\left[ (T_\lambda ^\vee ({\mathbb {C}}^2)^{[n]})^{\oplus g}\right] }{{{\,\mathrm{ch}\,}}_{ T}\Lambda _{-1}\left[ T^\vee _\lambda ({\mathbb {C}}^2)^{[n]}\right] }{{\,\mathrm{ch}\,}}_{ T}({\mathcal {P}}^\gamma _\mu ), \end{aligned}$$
(3.86)

where \({\mathcal {P}}^\gamma \) is a vector bundle over \(({\mathbb {C}}^2)^{[n]}\) whose fibers over closed points \([I]\in ({\mathbb {C}}^2)^{[n]}\) are isomorphic to permutation representations of \({\mathcal {S}}_n\).

By virtue of what we showed in Sect. 3.4, we expect our results to give a virtual refinement of the formulae found in [21, 55]. For the sake of simplicity, let us start from studying the case of a quiver consisting of only two gauge nodes and \(r=1\), corresponding to a complex curve \({\mathcal {C}}\) of genus \(g=0\). We already computed in Sect. 3.4 the partition function relative to any generic quiver of the type shown in Fig. 4, with \((r_0,r_1,\dots ,r_{s-1})=(r,0,\dots ,0)\). We will then be computing the generating function

$$\begin{aligned} Z^{(p_0,p_1)}_{\text {vir}}=\sum _{\mathbf{n }\in {\mathbb {Z}}^2_{\ge 0}}Z^{(p_0,p_1)}_{\mathbf{n }}\prod _{i=0}^1x_i^{n_i}=\sum _{\mathbf{n } \in {\mathbb {Z}}_{\ge 0}^2}{{\,\mathrm{ch}\,}}_{T}\chi _{T}^{\text {vir}}\left( {\mathcal {N}}_{1,[1^1],n, \gamma (\mathbf{n })},{\mathcal {L}}_{(p_0,p_1)}\right) \prod _{i=0}^1x_i^{n_i},\nonumber \\ \end{aligned}$$
(3.87)

where \(\gamma (\mathbf{n })\) is the ordered sequence determined by \(n_i\) determining the relevant quiver variety of numerical type \((1,{\hat{n}}_0,{\hat{n}}_1)\).

We will restrict our attention to \(\mathbf{p }=(p_0,0)\), in which case the restriction \({\mathcal {L}}_Z\) of \({\mathcal {L}}_{(p_0,0)}\) to the fixed point under \(T\curvearrowright {\mathcal {N}}_{1,[1^1],n,\gamma (\mathbf{n })}\) is

$$\begin{aligned} {\mathcal {L}}_Z=\left( \prod _{i=1}^{M_1}\prod _{j=1}^{\nu _i'}T_1^{-i+1}T_2^{-j+1} \right) ^{p_0}. \end{aligned}$$
(3.88)

The result obtained in Sect. 3.4 by means of SUSY localization then specializes in this case to the form (3.89):

$$\begin{aligned} Z^{(p_0,0)}_{\mathbf{n }}= & {} \sum _{\begin{array}{c} Z=(\nu ,\mu )\\ (|\nu |,|\mu |)=\gamma (\mathbf{n }) \end{array}}\frac{{{\,\mathrm{ch}\,}}_{T}{\mathcal {L}}_Z}{\Lambda _{-1} \left[ T_Z^{\text {vir}}{\mathcal {N}}_{1,[1^1],n,\gamma (\mathbf{n })}^\vee \right] }\nonumber \\= & {} \sum _{\begin{array}{c} Z=(\nu ,\mu )\\ (|\nu |,|\mu |)=\gamma (\mathbf{n }) \end{array}}\frac{{\mathcal {L}}_{\nu }(q_1,q_2) \tilde{{\mathcal {W}}}_{(\nu ,\mu )}(q_1,q_2)}{\Lambda _{-1}\left[ T_{{\tilde{Z}}} {\mathcal {M}}_{1,n_0}^\vee \right] }, \end{aligned}$$
(3.89)

with

$$\begin{aligned} {\mathcal {L}}_{\nu }(q_1,q_2)=\left( \prod _{i=1}^{M_1}\prod _{j=1}^{\nu _i'}q_1^{i-1} q_2^{j-1}\right) ^{p_0}, \end{aligned}$$
(3.90)

and

$$\begin{aligned} \tilde{{\mathcal {W}}}_{(\nu ,\mu )}=\prod _{i=1}^{M_1}\prod _{j=1}^{N_1} \frac{(1-q_1^{\mu _j-i}q_2^{j-\nu '_i-1})(1-q_1^{-i}q_2^{j-\mu '_i-1})}{(1-q_1^{\mu _j-i}q_2^{(j-\mu '_i-1})(1-q_1^{-i}q_2^{j-\nu '_i-1})} \prod _{i=1}^{M_1}\prod _{j=1}^{\nu '_i-\mu '_i} \frac{(1-q_1^{-i}q_2^{-j-\mu '_i})}{(1-q_1^{-1}q_2^{-1})},\nonumber \\ \end{aligned}$$
(3.91)

where as usual, \(q_1={{\,\mathrm{ch}\,}}_{ T}T_1\) and \(q_2={{\,\mathrm{ch}\,}}_{ T}T_2\).

In order to support our conjecture that the quiver we studied so far do indeed provide an ADHM-type construction for the nested Hilbert scheme of points on \({\mathbb {C}}^2\) we will show some relevant examples in the following. In the two-step quiver case this is true by a result of [54], which moreover implies that the non-abelian quiver provides an ADHM description for the moduli space of framed torsion-free flags of sheaves on \({\mathbb {P}}^2\). A very brief review of the result of [54] which are useful for what follows can be found in “Appendix C.” Even in the two-step case we can still compare the results coming from direct localization computations to the formulae in [21, 55]. In particular, since the nested Hilbert scheme of points is known to be non-smooth except for the case \((n_0,n_1)=(n,1)\) or \((n_0,n_1)=(n,0)\), the polynomials we get multiplied by the Nekrasov partition function order by order are expected to reproduce the modified Macdonald polynomials \({\tilde{H}}_\lambda (\mathbf{x };q,t)\) when \(n_1=1\). For the sake of ease of comparison, in what follows we will use the notation \({\mathcal {N}}(r,n_0,\dots ,n_{s-1})\), which is found in [21, 54], instead of \({\mathcal {N}}_{r,[r^1],n,\mu }\).

Example 2

If \(\mathbf{n }=(n,0)\) we need to compute the partition function for \({\mathcal {N}}(1,n,0)\), and obviously the partition function reproduces the result in Eq. (3.82), for \(g=0\).

Example 3

Take \(\mathbf{n }=(1,1)\), so that \({\mathcal {F}}(1,1,1)\simeq {\mathcal {N}}(1,2,1)\simeq {{\,\mathrm{Hilb}\,}}^{(1,2)}({\mathbb {C}}^2)\), [54]. We have two different choices for the fixed points:

(3.92)

and we have for the partition function

$$\begin{aligned}&Z_{\mathbf{n }}^{(1-g,0)}(\mathbf{x };q,t)=\sum _{\nu }Z_{\mathbf{n },\nu }^{(1-g,0)} (\mathbf{x };q,t)\nonumber \\&=x_0x_1\left\{ \sum _{(\nu ,\mu )} \frac{{\mathcal {L}}_{\nu }(q^{-1},t^{-1})\tilde{{\mathcal {W}}}_{(\nu ,\mu )} (q^{-1},t^{-1})}{\Lambda _{-1}\left[ T_{{\tilde{Z}}}{\mathcal {M}}_{1,n_0}^\vee \right] } \right\} \end{aligned}$$
(3.93)

with

$$\begin{aligned} \left\{ \begin{aligned}&Z_{\mathbf{n },2^1}^{(1-g,0)}(\mathbf{x };q,t) =\frac{{\mathcal {L}}_{2^1}(q^{-1},t^{-1})}{\Lambda _{-1}\left[ T_{2^1} {\mathcal {M}}_{1,n_0}^\vee \right] }(1+q)x_0x_1\\&Z_{\mathbf{n },1^2}^{(1-g,0)}(\mathbf{x };q,t) =\frac{{\mathcal {L}}_{1^2}(q^{-1},t^{-1})}{\Lambda _{-1}\left[ T_{1^2} {\mathcal {M}}_{1,n_0}^\vee \right] }(1+t)x_0x_1 \end{aligned}\right. \end{aligned}$$
(3.94)

By putting together with the previous example, we have that

$$\begin{aligned} \begin{aligned} Z_{|\mathbf{n }|=2}^{1-g,0}&=\sum _{\nu \in {\mathcal {P}}(2)} \frac{{\mathcal {L}}_{\nu }(q^{-1},t^{-1})}{\Lambda _{-1}\left[ T_{\nu } {\mathcal {M}}_{1,n_0}^\vee \right] }{\tilde{H}}_{\nu }(x_0,x_1;q,t)\\&=\sum _{\nu \in {\mathcal {P}}(2)}{\mathcal {H}}_\nu ^{g=0}(z,w) {\tilde{H}}_\nu (x_0,x_1;z^2,w^2) \end{aligned} \end{aligned}$$
(3.95)

We want to point out that the elliptic counterpart to the polynomials determined by \(\tilde{{\mathcal {W}}}_{(\nu ,\mu )}\) are the following

(3.96)

which obviously reduce to the corresponding modified Macdonald polynomials coefficients when \(\tau \rightarrow {\text{ i }}\infty \).

Example 4

Let’s now consider \(\mathbf{n }\) to be such that \(n_0+n_1=3\). The only quantity we need to compute is related to \(\mathbf{n }=(2,1)\), which corresponds to \({\mathcal {N}}(1,3,1)\). We have the following possibilities for the fixed points:

(3.97)

and

(3.98)

As in the previous example, we can exhibit explicitly the elliptic counterparts to these modified Macdonald polynomials, which read:

(3.99)

Example 5

As a final example of a smooth nested Hilbert scheme of points we will take \({\mathcal {N}}(1,4,1)\), so that the fixed points will be

(3.100)

by which we get

(3.101)

which again reproduce modified Macdonald polynomials which can be found tabulated in the mathematical literature. Their elliptic counterpart is now given by:

(3.102)

The following is the easiest example of a non-smooth nested Hilbert scheme, namely \({\mathcal {N}}(1,4,2)\), and we can see how in this case our computation does not reproduce the \(\chi _y\) genus of [21], hence the formulae of [55], giving instead their virtual generalization.

Example 6

Take \((n_0,n_1)=(4,2)\). The prescription for the fixed points gives us

(3.103)

by which we get

(3.104)

The polynomials above contain the coefficients for the modified Macdonald polynomials which in this case read

(3.105)

As a final remark let us point out that, even though the GLSM partition function is naturally computing virtual invariants, as the moduli space \({\mathcal {N}}(r,n_0,n_1)\) is in general a singular quasi-projective variety, [76], however one should be able to use equivariant localization to compute usual topological invariants also for singular varieties [77, 78].