Counting signed vexillary permutations
Introduction
Permutation pattern avoidance has been a popular line of research for many years. Denote the symmetric group on n elements by . We say that a permutation avoids a pattern if there do not exist indices such that if and only if . Let denote the set of permutations that avoid π. Two permutations are called Wilf equivalent if for all n. The study of the growth rate of and the study of nontrivial Wilf equivalence classes have been fruitful. One of the most famous examples of Wilf equivalence classes is all permutation patterns of length 3. Specifically, for any , , the nth Catalan number.
Likewise, the set of permutations avoiding 1234 and the set of permutations avoiding 2143 have been traditionally well-studied and enjoy nice combinatorial properties. Their permutation matrices are shown in Fig. 1.
A permutation w avoids 1234 if and only if its shape under RSK has at most 3 columns, by Greene's theorem (see for example [10]). With some further tools in the theory of symmetric functions, 1234-avoiding permutations can be enumerated as follows: This enumeration appeared in many previous works including [6], [7] and [4] and is now an exercise in chapter 7 of [10]. A permutation that avoids 2143 is called vexillary. Vexillary permutations can also be characterized as the permutations whose Rothe diagram, up to a permutation of its rows and columns, is the diagram for a partition [8]. Moreover, their associated Schubert polynomials are flag Schur functions [8]. West [12] showed that for any n, so 1234 and 2143 are Wilf equivalent.
The notion of Wilf equivalence can be naturally generalized to signed permutations. The signed permutation group , also known as the Weyl group of type or , consists of permutations w on such that for all . We say that avoids if the natural embedding of w into avoids π in the sense of permutation pattern avoidance. For example, given by , , , contains 231 and does not contain 123, as the natural embedding sends w to , and 3142 contains 231 and does not contain 123. As a warning, this definition of pattern containment is not equivalent to a Weyl group element w of type B avoiding a type A pattern, in the sense of root system pattern avoidance defined by Billey and Postnikov [3] to study the smoothness of Schubert varieties.
In particular, let us define i.e., the set of signed permutations avoiding 1234 and the set of signed permutations avoiding 2143 respectively, which are the main objects of interest in this paper.
Analogously, and have very nice properties. In particular, the enumeration result where is the jth Catalan number, is given by Egge [5], using techniques involving RSK and jeu-de-taquin. Geometric and combinatorial properties of signed permutations avoiding 2143, which are also called vexillary signed permutations, are studied by Anderson and Fulton [1]. They conjectured that . The main result of this paper is to answer this conjecture positively.
In fact, there are more similarities between the structures of signed permutations avoiding 1234 and signed permutations avoiding 2143. For , and , define Theorem 1.1 For , .
By summing over , we obtain the aforementioned conjecture by Anderson and Fulton [1] as a corollary. Corollary 1.2 For ,
As pointed out by Christian Gaetz, Theorem 1.1 also implies a direct analogue of Corollary 1.2 for type D. Recall that the Weyl group of type can be realized as an order 2 subgroup of the Weyl group of type . Specifically, We then say that avoids a pattern π if avoids π. And as an analogous notation, let denote the set of that avoids π. Similarly, elements in are called vexillary in type D [2]. By summing the equality in Theorem 1.1 over with even, we obtain the analogous enumeration result in type D. Corollary 1.3 For , .
The main tool that we use in the proof of Theorem 1.1 is the idea of generating trees developed by West [12] to show that . A generating tree is a rooted labeled tree for which the label at a vertex determines its descendants (their number and their labels). The generating trees considered by West have vertices that correspond to permutations avoiding a fixed pattern π. The descendants of a vertex corresponding to the permutation correspond to permutations formed by inserting a new largest element to some location in w (so that π is still avoided). The usefulness of such generating trees stems in part from the fact that it is often possible to present an isomorphic tree with vertices labeled by only a few permutation statistics, with a simple enough succession rule to be fit for further analysis. In the case of versus , West was able to find a simple description of both trees and observed that the two are naturally isomorphic, thus proving bijectively.
There are two main difficulties in proving the simple-looking theorem (Theorem 1.1). First, as pointed out by Anderson and Fulton [1], the bijection between and provided by West [12] does not preserve whether the permutation equals its reverse complement or not, suggesting a more careful choice of statistics for the generating trees, described in Section 2. Second, the generating trees for and turn out to be far from isomorphic unlike the case of versus so we finish the proof by using certain generating functions in Section 3. We end in Section 4 with discussion on open problems.
Section snippets
Generating trees for 1234 and 2143 avoiding permutations
We will start working towards an explicit generating tree for and . Throughout the section, a signed permutation should be visualized by a point graph, where the x-axis corresponds to the indices , , and the y-axis corresponds to the images . As a one-line notation, we will denote w by in a nonstandard way, for reasons soon to be clear. A visualization of is shown in Fig. 2.
We first prove some simple
Finishing the proof
Proposition 2.6 and Proposition 2.7 allow us to translate the questions of enumerating and to questions of enumerating lattice paths in the integer lattice with specified rules. Respectively, let be the set of all lattice paths specified by the succession rule in Proposition 2.6 and let be the set of all lattice paths specified by the succession rule in Proposition 2.7. We allow arbitrary starting point for those paths with and besides those
Open questions
There are still many interesting questions to be asked.
Firstly, the proof provided in Section 3 is semi-bijective. With recursive formulas provided in Lemma 3.4 and Lemma 3.5, we are able to obtain the equality of . However, is there an explicit bijection between paths in and that is length-preserving?
Secondly, for a fixed , it is desirable to obtain an explicit formula for the generating function for either . The
Acknowledgments
This research was carried out as part of the 2019 Summer Program in Undergraduate Research (SPUR) of the MIT Mathematics Department. The authors thank Prof. Alex Postnikov for suggesting the project and Christian Gaetz, Prof. Ankur Moitra and Prof. David Jerison for helpful conversations.
References (12)
- et al.
Smoothness of Schubert varieties via patterns in root subsystems
Adv. Appl. Math.
(2005) Symmetric functions and P-recursiveness
J. Comb. Theory, Ser. A
(1990)- et al.
Mahonian pairs
J. Comb. Theory, Ser. A
(2012) Generating trees and the Catalan and Schröder numbers
Discrete Math.
(1995)- et al.
Vexillary signed permutations revisited
- et al.
Vexillary elements in the hyperoctahedral group
J. Algebraic Comb.
(1998)