A PIERI-TYPE FORMULA FOR ISOTROPIC FLAG MANIFOLDS

NANTEL BERGERON AND FRANK SOTTILE

Abstract. We give the formula for multiplying a Schubert class on an odd orthogonal or symplectic flag manifold by a special Schubert class pulled back from a Grassmannian of maximal isotropic subspaces. This is also the formula for multiplying a type B (respectively, type C) Schubert polynomial by the Schur P -polynomial p_{m} (respectively, the Schur Q-polynomial q_{m}). Geometric constructions and intermediate results allow us to ultimately deduce this from formulas for the classical flag manifold. These intermediate results are concerned with the Bruhat order of the Coxeter group B_{∞}, identities of the structure constants for the Schubert basis of cohomology, and intersections of Schubert varieties. We show these identities follow from the Pieri-type formula, except some ‘hidden symmetries’ of the structure constants. Our analysis leads to a new partial order on the Coxeter group B_{∞} and formulas for many of these structure constants.

Contents

Introduction 1 1. Statement of results 2 2. Orders on B_{∞} 8 2.1. The 0-Bruhat order 10 2.2. A monoid for the Lagrangian order 14 3. Isotropic flag manifolds and maximal Grassmannians 16 3.1. Schubert varieties 17 4. Identities of structure constants 20 5. More identities 24 5.1. Product decomposition 24 5.2. Some hidden symmetries 24 5.3. More hidden symmetries 27 6. Minimal permutations and labeled réseaux 30 6.1. Minimal permutations 30 6.2. The Grassmann-Bruhat order on S_{∞} 31 6.3. The labeled Lagrangian order 33 6.4. The Lagrangian réseau 34

Date: 2 July 1998. 1991 Mathematics Subject Classification. 14M15, 05E15, 05E05, 06A07, 14N10. Key words and phrases. Pieri formula, isotropic flag manifold, Bruhat order, Schubert variety,

Lagrangian Grassmannian, Schubert polynomial, Schur P -function. First author supported in part by NSERC and CRM grants. Second author supported in part by NSERC grant OGP0170279. research at MSRI supported by NSF grant DMS-9022140.

7. The Pieri-type formula 37 7.1. Case δ(ζ) = 1 40 7.2. Case δ(ζ) = 0 40 References 45

Introduction

The cohomology of a flag manifold G/B has an integral basis of Schubert classes 𝔖_{w} indexed by elements w of the Weyl group of G. The algebraic structure of these rings is known [9] with respect to a monomial basis, and there are methods (Schubert polynomials) for expressing the 𝔖_{w} in terms of this basis [6, 7, 12, 17, 19, 25, 29]. Moreover, their multiplicative structure with respect to the Schubert basis is determined by Chevalley’s formula [10]. Despite this, it remains an open problem to give a closed or bijective formula for the integral structure constants c_{u v}^{w} defined by the identity

𝔖_{u} · 𝔖_{v} = ∑_{w} c_{u v}^{w} 𝔖_{w}.

These c_{u v}^{w} are non-negative as they count the flags in a suitable triple intersection of Schubert varieties. They are expected to be related to the enumeration of chains in the Bruhat order of the Weyl group (see [3] and the references therein).

Of particular interest are Pieri-type formulas which describe the constants c_{u v}^{w} when 𝔖_{v} is a special Schubert class pulled back from a Grassmannian projection (G/P, P maximal parabolic), as these determine the ring structure for the cohomology of G/P when P is any parabolic subgroup. When G is SL_{n}ℂ, a Pieri-type formula for multiplication by a special Schubert class was described [25] in terms of the Weyl group element wu^{−1}. A formula in terms of chains in the Bruhat order was conjectured [1] and given a geometric proof [31]. Our main results are the analogous formulas when G is Sp_{2n}ℂ or So_{2n+1}ℂ and 𝔖_{v} is a special Schubert class pulled back from a Grassmannian of maximal isotropic subspaces. These are common generalizations of the Pieri-type formulas for SL_{n}ℂ, Chevalley’s formula, and Pieri-type formulas for Grassmannians of maximal isotropic subspaces [8].

Our proof uses results on the Bruhat order, identities of these structure constants, a decomposition of intersections of Schubert varieties, and formulas in the cohomology of the SL_{n}ℂ-flag manifold to explicitly determine a triple intersection of Schubert varieties. This shows the coefficients in the Pieri-type formula are the intersection number of a linear space with a collection of quadrics. Some intermediate results, including a fundamental identity and some additional ‘hidden symmetries’ of the structure constants, are deduced from constructions on SL_{n}ℂ-flag manifolds [3]. This analysis leads to other results, including a new partial order on the infinite Coxeter group B_{∞} and a monoid for chains in this order as in [4]. We show how the Pieri-type formula implies our fundamental identity, use the identities to express many structure constants in terms of the Littlewood-Richardson coefficients for the multiplication of Schur P – (or Q–) functions [33], and apply the hidden symmetries to the enumeration of chains in the Bruhat order.

1. Statement of results

Schubert classes in the cohomology of the flag manifolds So_{2n+1}ℂ/B and Sp_{2n}ℂ/B form integral bases indexed by elements of the Weyl group B_{n}. We represent B_{n} as the group of permutations w of {−n, …, −2, −1, 1, …, n} satisfying w(−a) = −w(a) for 1 ≤ a ≤ n. Let 𝔅_{w} denote the Schubert class indexed by w ∈ B_{n} in H^{∗}So_{2n+1}ℂ/B and ℭ_{w} that in H^{∗}Sp_{2n}ℂ/B. The degree of these classes is 2 · ℓ(w), where the length ℓ(w) of w is

#{0 < i < j ≤ n | w(i) > w(j)} + ∑_{i>0>w(i)} |w(i)|.

For an integer i, let \overline{ı} denote −i. For each 1 ≤ m ≤ n, define v_{m} ∈ B_{n} by

\overline{m} = v_{m}(1) < 0 < v_{m}(2) < ⋯ < v_{m}(n).

This indexes a (maximal isotropic) special Schubert class in either cohomology ring, written as p_{m} := 𝔅_{v_{m}} and q_{m} := ℭ_{v_{m}}. We state the Pieri-type formula for the products 𝔅_{w} · p_{m} and ℭ_{w} · q_{m} in terms of chains in the Bruhat order on B_{n}. For this, we need a definition.

Definition 1.1. The 0-Bruhat order ≤_{0} on B_{n} is defined recursively as follows: u⋖_{0}w is a cover in the 0-Bruhat order if and only if

(1) ℓ(u) + 1 = ℓ(w), and

(2) u^{−1}w is a reflection of the form (\overline{ı}, i) or (\overline{ı}, j)(\overline{ȷ}, i) for some 0 < i < j ≤ n.

Chevalley’s formula [10] may be stated as follows:

\begin{aligned}𝔅_{u} · p_{1} & = & ∑_{u⋖_{0}w} 𝔅_{w} \\ ℭ_{u} · q_{1} & = & ∑_{u⋖_{0}w} χ(u^{−1}w)ℭ_{w}, \end{aligned} (1)

where χ(u^{−1}w) is the number of transpositions in the reflection u^{−1}w.

We enrich the 0-Bruhat order in two complementary ways. Write the two types of covers in the 0-Bruhat order as u ⋖_{0} (\overline{β}, β)u and u ⋖_{0} (\overline{β}, \overline{α})(α, β)u where 0 < α < β ≤ n. The labeled 0-Bruhat réseau is a labeled directed multigraph with vertex set B_{n} and labeled edges between covers in the 0-Bruhat order given by the following rule: If u ⋖_{0} (\overline{β}, β)u, then a single edge is drawn with label β. If u ⋖_{0} (\overline{β}, \overline{α})(α, β)u, then two edges are drawn with respective labels \overline{α} and β. Thus if u ⋖_{0} w, then χ(u^{−1}w) counts the edges from u to w in this 0-Bruhat réseau. The labeled 0-Bruhat order is obtained from this réseau by removing edges with negative integer labels.

Given a (saturated) chain γ in either of these structures, let end(γ) denote the endpoint of γ. A peak in a chain γ is an index i ∈ {2, …, m−1} with a_{i−1} < a_{i} > a_{i+1}, where a_{1}, a_{2}, …, a_{m} is the sequence of edge labels in γ. A descent is an index i < m with a_{i} > a_{i+1} and an ascent is an index i < m with a_{i} < a_{i+1}.

Theorem A. (Pieri-type formula) Let u ∈ B_{n} and m > 0. Then I. (Odd-orthogonal Pieri-type formula)

𝔅_{u} · p_{m} = ∑ 𝔅_{end(γ)},

the sum over all chains γ in the labeled 0-Bruhat order of B_{n} which begin at u, have length m, and no peaks.

II. (Symplectic Pieri-type formula)

ℭ_{u} · q_{m} = ∑ ℭ_{end(γ)},

(a) the sum over all chains γ in the labeled 0-Bruhat réseau of B_{n} which begin at u, have length m, and no descents.

(b) the same sum, except with no ascents.

This generalizes Chevalley’s formula and the Pieri-type formulas for SL_{n}ℂ/B, which are expressed in [31] as a sum of certain labeled chains in the Bruhat order on the symmetric group S_{n} with no ascents/no descents. The duality of these two formulas, one in terms of peaks for an order, and the other in terms of descents/ascents for an enriched structure on that order has connections with other dualities in combinatorics. These include Fomin’s duality of graded graphs [15, 16] and Stembridge’s theory of enriched P -partitions [34], where peak and descent sets play a complementary role. These relations are explored in [2], which extends the theory developed in [5] to the ordered structures of this manuscript.

Example 1.2. Represent permutations w ∈ B_{3} by their values w(1) w(2) w(3). Consider the products 𝔅_{3\overline{1}2} · p_{2} and ℭ_{3\overline{1}2} · q_{2}. Figure 1 shows the part of the 0-Bruhat réseau of height 2 above 3\overline{1}2 in B_{3}. (Erase edges with negative labels to obtain its

Figure 1. Chains above 3\overline{1}2

analog in the 0-Bruhat order.) Chains of length 2 are peakless, so by Theorem A I, we have

𝔅_{3\overline{1}2} · p_{2} = 2 𝔅_{\overline{3} \overline{2}1} + 𝔅_{2\overline{3}1} + 𝔅_{3\overline{2} \overline{1}} + 𝔅_{\overline{1} \overline{3}2}.

Every chain in Figure 1 with increasing labels may be paired with a chain with decreasing labels having the same underlying permutations, and this pairing exhausts all chains. Thus, by Theorem A II, we have

ℭ_{3\overline{1}2} · q_{2} = 2 ℭ_{\overline{3} \overline{2}1} + 2 ℭ_{2\overline{3}1} + ℭ_{3\overline{2} \overline{1}} + ℭ_{\overline{1} \overline{3}2}.

If λ is a strict partition (decreasing integral sequence n ≥ λ_{1} > λ_{2} > ⋯ > λ_{k} > 0), then λ determines a unique Grassmannian permutation v(λ) ∈ B_{n} where v(i) = \overline{λ_{i}} for i ≤ k and 0 < v(k + 1) < ⋯ < v(n). If k = 1 and λ_{1} = m, then v_{m} = v(λ). The Schubert classes P_{λ} := 𝔅_{v(λ)} and Q_{λ} := ℭ_{v(λ)} are pullbacks of Schubert classes from the Grassmannians of maximal isotropic subspaces So_{2n+1}ℂ/P_{0} and Sp_{2n}ℂ/P_{0}, where P_{0} is the maximal parabolic associated to the simple root of exceptional length.

Formulas for products of these P - and Q-classes are known [33] as these classes are specializations of Schur P - and Q-functions [21, 28]. Our proof of Theorem A uses identities among the structure constants b_{u λ}^{w} and c_{u λ}^{w} defined by the following formulas.

𝔅_{u} · P_{λ} = ∑_{w} b_{u λ}^{w} 𝔅_{w} and ℭ_{u} · Q_{λ} = ∑_{w} c_{u λ}^{w} ℭ_{w}

If u, w, v(λ) ∈ B_{n}, then these constants do not depend upon n.

Iterating Chevalley’s formula (1) shows that if either of b_{u λ}^{w} or c_{u λ}^{w} is non-zero, then u <_{0} w and ℓ(w) − ℓ(u) equals |λ|, the sum of the parts of λ. In fact the constant b_{u λ}^{w} determines and is determined by the constant c_{u λ}^{w}: Let s(w) count the sign changes ({i | i > 0 > w(i)}) in w. Then the map ℭ_{w} ↦ 2^{s(w)}𝔅_{w} embeds H^{∗}Sp_{2n}ℂ/B into H^{∗}So_{2n+1}ℂ/B and induces an isomorphism of their rational cohomology rings. Thus it suffices to work in H^{∗}Sp_{2n}ℂ/B. This is fortunate, as a key geometric result, Theorem 3.4(2), holds only for Sp_{2n}ℂ/B.

Let f_{u}^{w} count the saturated chains in the interval [u, w]_{0} and g_{u}^{w} count the saturated chains in the réseau [u, w]_{0}. Iterating Chevalley’s formula (1) with u = e, the identity permutation, we obtain the following expressions.

p_{1}^{m} = ∑_{|λ|=m} f_{e}^{v(λ)} P_{λ} and q_{1}^{m} = ∑_{|λ|=m} g_{e}^{v(λ)} Q_{λ}

Multiplying the first expression by 𝔅_{u} and collecting the coefficients of 𝔅_{w} in the resulting expansion (likewise for the second expression) gives the following proposition.

Proposition 1.3. Let u, w ∈ B_{n}. Then

f_{u}^{w} = ∑_{|λ|=ℓ(w)−ℓ(u)} f_{e}^{v(λ)} b_{u λ}^{w}  and  g_{u}^{w} = ∑_{|λ|=ℓ(w)−ℓ(u)} g_{e}^{v(λ)} c_{u λ}^{w}.

Theorem A and Proposition 1.3 show a close connection between chains in the 0-Bruhat order/réseau and the structure constants b_{u λ}^{w} and c_{u λ}^{w}. This justifies an elucidation of the basic properties of the 0-Bruhat order and réseau, which we do in Sections 2 and 6. These structures have a remarkable property and there are related fundamental identities among the structure constants.

Theorem B. Suppose u <_{0} w and x <_{0} z in B_{n} with wu^{−1} = zx^{−1}. Then

(1) The map v ↦ vu^{−1}x induces an isomorphism of labeled intervals in the 0-Bruhat order and 0-Bruhat réseau [u, w]_{0} ⟶^{∼} [x, z]_{0}.

(2) For any strict partition λ,

b_{u λ}^{w} = b_{x λ}^{z}  and  c_{u λ}^{w} = c_{x λ}^{z}.

We prove Theorem B(1) in Section 2.1 using combinatorial methods. Theorem B(2) is a consequence of a geometric result (Theorem 3.3) proven in Section 4. Both parts of Theorem B are key to our proof of the Pieri-type formula. Interestingly, the Pieri-type formula and Theorem B(1) together imply Theorem B(2):

For any composition α = (α_{1}, …, α_{s}) with each α_{i} ≥ 0, let p_{α} := p_{α_{1}} ⋯ p_{α_{s}}, q_{α} := q_{α_{1}} ⋯ q_{α_{s}}, and I(α) := {α_{1}, α_{1} + α_{2}, …, α_{1} + ⋯ + α_{s−1}}. The peak set of a (maximal) chain in a labeled order is the set of indices of peaks in the chain. Given a chain in a labeled réseau, its descent set (respectively ascent set) is the set of indices of descents (respectively ascents) in the chain.

Corollary 1.4. Let u, w, x, z ∈ B_{n}.

(1) Let α be any composition. Then the coefficient of 𝔅_{w} in the product 𝔅_{u} · p_{α} is the number of chains in the interval [u, w]_{0} in the labeled 0-Bruhat order with peak set contained in I(α).

(2) Let α be any composition. Then the coefficient of ℭ_{w} in the product ℭ_{u} · q_{α} is the number of chains in the interval [u, w]_{0} in the labeled 0-Bruhat réseau with descent set contained in I(α). This is also the number with ascent set contained in I(α).

(3) Suppose the Pieri-type formula (Theorem A) holds. Then the intervals [u, w]_{0} and [x, z]_{0} have the some number of chains with peak set I(α) for every composition α if and only if for every strict partition λ, b_{u λ}^{w} = b_{x λ}^{z}. The same statement holds for ascent/descent sets for chain in the réseaux and the coefficients c_{u λ}^{w}, c_{x λ}^{z}. In particular, Theorem B(1) implies Theorem B(2).

Moreover, the numbers in 1 and 2 depend only upon the multiset {α_{1}, …, α_{s}}.

Parts 1 and 2 follow from Theorem A. For 3, note that the Schur P -polynomials (respectively Q-polynomials) are linear combinations of the p_{α} (respectively the q_{α}) [27, III.8.6]. This linear combination gives a formula for b_{u λ}^{w} (respectively c_{u λ}^{w}) in terms of chains with given peak sets (respectively, given ascent/descent sets).

Let ζ ∈ B_{n}. By Theorem B(1), we may define η ⪯ ζ if there is a u ∈ B_{n} with u ≤_{0} ηu ≤_{0} ζu and L(ζ) := ℓ(ζu) − ℓ(u) whenever u ≤_{0} ζu. Then (B_{n}, ≺) is a graded partial order with rank function L(·). By the identity of Theorem B(2), we may define b_{λ}^{ζ} := b_{u λ}^{ζu} and c_{λ}^{ζ} := c_{u λ}^{ζu} for any u ∈ B_{n} with u ≤_{0} ζu and |λ| = L(ζ).

These coefficients satisfy one obvious identity, c_{λ}^{ζ} = c_{λ}^{ζ^{−1}}, as c_{u v}^{w} = c_{ω_{0}u v}^{ω_{0}w} where ω_{0} ∈ B_{n} is the longest element. They also satisfy two others, which we call hidden symmetries. Let ρ ∈ B_{n} be the permutation defined by ρ(i) = i − 1 − n for 1 ≤ i ≤ n. Then ρ is the element with largest rank in (B_{n}, ≺). Let γ ∈ B_{n} be defined by γ(1) = 2, γ(2) = 3, …, γ(n) = 1, so that γ = (\overline{1}, \overline{2}, …, \overline{n})(1, 2, …, n).

Theorem C. For any ζ ∈ B_{n},

(1) L(ζ) = L(ρζρ) and for any strict partition λ, we have b_{λ}^{ζ} = b_{λ}^{ρζρ} and c_{λ}^{ζ} = c_{λ}^{ρζρ}.

(2) If a · ζ(a) > 0 for all a, then L(ζ) = L(γζγ^{−1}) and for any strict partition λ, we have b_{λ}^{ζ} = b_{λ}^{γζγ^{−1}} and c_{λ}^{ζ} = c_{λ}^{γζγ^{−1}}.

We prove a strengthening of Theorem B, relaxing the condition of equality of wu^{−1} and zx^{−1} to that of shape equivalence. Permutations η, ζ ∈ B_{n} are shape equivalent if there exist sets I : 0 < i_{1} < ⋯ < i_{s} ≤ n and J : 0 < j_{1} < ⋯ < j_{s} ≤ n such that η acts as the identity on {1, …, n} \ I, ζ acts as the identity on {1, …, n} \ J, and η(i_{k}) = i_{l} if and only if ζ(j_{k}) = j_{l}.

Theorems B (the stronger version) and C allow us to determine many of the constants b_{u λ}^{w} and c_{u λ}^{w}, showing they equal certain Littlewood-Richardson coefficients b_{μ λ}^{κ} and c_{μ λ}^{κ}for Schur P - and Q-functions. These are defined by the identities

P_{μ} · P_{λ} = ∑_{κ} b_{μ λ}^{κ} P_{κ} and Q_{μ} · Q_{λ} = ∑_{κ} c_{μ λ}^{κ} Q_{κ}.

A combinatorial formula for these coefficients was given by Stembridge [33].

Definition 1.5. Let μ, κ be strict partitions with μ ⊂ κ. We say that a permutation ζ ∈ B_{n} has skew shape κ/μ if

(1) Either ζ or ζ^{ρ} is shape equivalent to v(κ)v(μ)^{−1}, or

(2) If a · ζ(a) > 0 for all a, and one of ζ, ζ^{γ}, ζ^{γ^{2}}, …, ζ^{γ^{n−1}} is shape equivalent to v(κ)v(μ)^{−1}.

Corollary 1.6. If u ≤_{0} w are permutations in B_{n} and wu^{−1} has a skew shape κ/μ, then for any strict partition λ we have

b_{u λ}^{w} = b_{μ λ}^{κ}  and  c_{u λ}^{w} = c_{μ λ}^{κ}.

We call the partial order ≺ the Lagrangian order and transfer the labeling from the 0-Bruhat order to obtain the labeled Lagrangian order. In the same fashion, we may transfer the labeling and multiple edges of the 0-Bruhat réseau to (B_{n}, ≺), obtaining the (labeled) Lagrangian réseau. By Corollary 1.4(3), Theorem C has a purely enumerative corollary.

Corollary 1.7. For any ζ ∈ B_{n},

(1) For any subset S of {2, …, L(ζ) − 1}, the intervals [e, ζ]_{≺} and [e, ρζρ]_{≺} in the Lagrangian order have the same number of chains with peak set S.

(2) For any subset S of {1, …, L(ζ) − 1}, the intervals [e, ζ]_{≺} and [e, ρζρ]_{≺} in the Lagrangian réseau order have the same number of chains with descent set S and the same number of chains with ascent set S, and these two numbers are equal.

(3) If a · ζ(a) > 0 for all a, then the same is true for [e, ζ]_{≺} and [e, γζγ^{−1}]_{≺}.

In general, [e, ζ]_{≺} ≄ [e, ρζρ]_{≺}, (See Figure 6 in Example 5.2) and if a · ζ(a) > 0 for all a, then in general, [e, ζ]_{≺} ≄ [e, γζγ^{−1}]_{≺}. (See Figure 7 in Example 5.12.)

Let supp(ζ) := {a > 0 | ζ(a) ≠ a}, the support of ζ. A permutation ζ ∈ B_{n} is reducible if it has a non-trivial factorization ζ = η · ξ with L(ζ) = L(η) + L(ξ) where η and ξ have disjoint supports (η ·ξ = ξ ·η). If η ·ξ = ξ ·η with L(ξ ·η) = L(η)+L(ξ), then η · ξ is a disjoint product. Permutations w ∈ B_{n} have unique factorizations into irreducibles.

We represented B_{n} ⊂ S_{±[n]}, the group of permutations of {−n, …, −1, 1, …, n}. We also have S_{n} ↪ B_{n}, and the image consists of those ζ with a·ζ(a) > 0 for every a: For η ∈ S_{n}, let \overline{η} ∈ S_{−[n]} be the permutation such that \overline{η}(−i) = −η(i). Then η\overline{η} ∈ B_{n}. For ζ ∈ B_{n}, define δ(ζ) = 1 if ζ is in the image of S_{n}, and δ(ζ) = 0 otherwise.

In Section 6.1 we establish the following result.

Lemma 1.8. Let ζ ∈ B_{n} and suppose ζ is irreducible. Then L(ζ) ≥ #supp(ζ)−δ(ζ). If L(ζ) = #supp(ζ) − δ(ζ), then either

(1) we have δ(ζ) = 1 and there exists a cycle η ∈ S_{n} with ζ = η\overline{η}, or

(2) we have δ(ζ) = 0 and ζ is a single cycle in S_{±[n]}.

Definition 1.9. If every irreducible factor η of ζ satisfies L(η) = #supp(η) − δ(η), then we say that ζ is minimal.

If ζ ∈ B_{n} is minimal, then set

θ(ζ) := 2^{#{irreducible factors of ζ}−1},
χ(ζ) := 2^{#{irreducible factors η of ζ with δ(η) = 1}}.

If ζ is not minimal, then set θ(ζ) = χ(ζ) = 0.

We state the Pieri-type formula in terms of the permutation wu^{−1}.

Theorem D. Let u, w ∈ B_{n} and m ≤ n. Then

𝔅_{u} · p_{m} = ∑ θ(wu^{−1})𝔅_{w}  and  ℭ_{u} · q_{m} = ∑ χ(wu^{−1})ℭ_{w},

the sum over all w ∈ B_{n} with u ≤_{0} w and ℓ(w) − ℓ(u) = m.

This is similar to the form of the Pieri-type formula for SL_{n}ℂ/B in [25], which is in terms of the cycle structure of the permutation wu^{−1}. This also generalizes the form of the the Pieri formula for Grassmannians of maximal isotropic subspaces [8]. Our proof for Sp_{2n}ℂ/B shows these multiplicities to arise from the intersection of a linear subspace of ℙ^{2n−1} with a collection of quadrics, one for each irreducible factor η of wu^{−1} with δ(η) = 1, similar to the proof of the Pieri-type formula for maximal isotropic Grassmannians in [32]. We relate the two formulations (Theorems A and D) of the Pieri-type formula in sections 6.3 and 6.4.

Example 1.10. We return to Example 1.2. For u = 3\overline{1}2 and w equal to each of \overline{3} \overline{2}1, 2\overline{3}1, 3\overline{2} \overline{1}, and \overline{1} \overline{3}2 in turn, wu^{−1} is the permutation in S_{±[3]}:

(12)(\overline{1} \overline{2}) (3\overline{3}), (132)(\overline{1} \overline{3} \overline{2}), (12\overline{1} \overline{2}), and (13\overline{1} \overline{3}).

As permutations in B_{3}, the first has 2 irreducible factors, for these we have δ((12)(\overline{1} \overline{2})) = 1 and δ((3\overline{3})) = 0. The other permutations are irreducible with δ((132)(\overline{1} \overline{3} \overline{2})) = 1 and δ((12\overline{1} \overline{2})) = δ((13\overline{1} \overline{3})) = 0. Thus the values of θ are 2, 1, 1, 1 and of χ are 2, 2, 1, 1, which shows the two forms of the Pieri-type formula agree on this example.

We remark that the last two permutations, (12\overline{1} \overline{2}) and (13\overline{1} \overline{3}), are shape equivalent.

While we use the cohomology rings of complex varieties, our results and methods are valid for the Chow rings [18] and l-adic (étale) cohomology [11] of these same varieties over any field not of characteristic 2.

This paper is organized as follows: Section 2 contains basic combinatorial definitions and properties of the Bruhat order on B_{∞} analogous to those of the symmetric group established in [3, 4]. Section 3 contains the basic geometric definitions. In Section 4, we use geometry to establish the main identity, Theorem B(2). In Section 5, we establish additional geometric results and prove Theorem C. In Section 6, we establish further combinatorial properties of the Lagrangian order and réseau needed for the proof of the Pieri-type formula, which is given in Section 7.

2. Orders on B_{∞}

We derive the basic properties of the 0-Bruhat order on B_{∞} analogous to properties of the k-Bruhat order on S_{∞}. Further properties are developed in Section 6.

Let #S be the cardinality of a finite set S. For an integer j, its absolute value is |j| and let \overline{ȷ} := −j. Likewise, for a set P of integers, define \overline{P} := \{\overline{ȷ} | j ∈ P \} and ±P := P ∪ \overline{P}. Set [n] := \{1, …, n\} and let S_{±[n]} be the group of permutations of ±[n]. Let e be the identity permutation in S_{±[n]} and ω_{0} the longest element in S_{±[n]}: ω_{0}(i) = \overline{ı}. Then B_{n} is the subgroup of S_{±[n]} for which ω_{0}wω_{0} = w and ω_{0} ∈ B_{n}. We also have B_{n} ⊂ S_{[\overline{n},n]}, the symmetric group on [\overline{n}, n] := ±[n] ∪ \{0\}. We refer to elements of these groups as permutations. Permutations w ∈ B_{n} are often represented by their values w(1) w(2) … w(n). For example, 2 4 \overline{3} \overline{1} ∈ B_{4}. The length ℓ(w) of w ∈ B_{n} is

ℓ(w) = #{0 < i < j | w(i) > w(j)} + ∑_{i>0>w(i)} |w(i)|.

Thus ℓ(2 4 \overline{3} \overline{1}) = 4 + 4 = 8. Note that ω_{0} is the longest element in B_{n}.

An important class of permutations are the Grassmannian permutations, those v ∈ B_{n} for which v(1) < v(2) < ⋯ < v(n). Such a permutation is determined by its initial negative values. If v(k) < 0 < v(k + 1), define λ(v) to be the decreasing sequence \overline{v(1)} > \overline{v(2)} > ⋯ > \overline{v(k)}. Note that ℓ(v) = \overline{v(1)} + ⋯ + \overline{v(k)} =: |λ(v)|. Likewise, given a decreasing sequence μ of positive integers (a strict partition) with n ≥ μ_{1}, let v(μ) be the Grassmannian permutation with λ(v(μ)) = μ. We write μ ⊂ λ for strict partitions μ, λ if μ_{i} ≤ λ_{i} for all i, equivalently, if v(μ) ≤_{0} v(λ).

The inclusion ±[n] ↪ ±[n+1] induces inclusions B_{n} ↪ B_{n+1} and S_{±[n]} ↪ S_{±[n+1]}. Define B_{∞} := ⋃_{n} B_{n} and S_{±∞} := ⋃_{n} S_{±[n]}.

In this representation, B_{∞} has three types of reflections, which are, as elements of S_{±∞}:

\begin{aligned}t_{i j} & := & (\overline{ȷ}, \overline{ı})(i, j) \\ t_{j} & := & (\overline{ȷ}, j) \\ t_{\overline{ı} j} & := & (\overline{ȷ}, i)(\overline{ı}, j) \end{aligned} for 0 < i < j.

These reflections act on positions on the right and on values on the left. The Bruhat order on B_{∞} is defined by its covers: u⋖w if ℓ(u)+1 = ℓ(w) and u^{−1}w is a reflection.

For each k = 0, 1, …, define the k-Bruhat order (on B_{n} or B_{∞}) by its covers: Set u ⋖_{k} w if u ⋖ w and

u^{−1}w is one of \{ \begin{matrix}t_{i j} & with i ≤ k < j, or \\ t_{j} & with k < j, or \\ t_{\overline{ı} j} & with k < j. \end{matrix}

For example, Figure 2 shows all covers w ∈ B_{4} of u = 2 4 \overline{3} \overline{1}, the reflection u^{−1}w, and for which k this is a cover in the k-Bruhat order.

Figure 2. Covers of 2 4 \overline{3} \overline{1}

2.1. The 0-Bruhat order. While these orders are analogous to the k-Bruhat orders on S_{∞} [3, 4, 26, 31], only the 0-Bruhat order on B_{∞} enjoys most properties of the k-Bruhat orders on S_{∞}. This is because the 0-Bruhat order is an induced suborder of the 0-Bruhat order on S_{±∞}.

The length l(w) of a permutation w ∈ S_{±∞} counts the inversions of w:

l(w) := #{i < j | w(i) > w(j)}.

The Bruhat order (◁) on S_{±∞} is defined by its covers: u ◁· w if and only if wu^{−1} is a transposition and l(w) = l(u) + 1. If k ∈ ℤ, this is a cover (written ◁·_{k}) in the k-Bruhat order (◁_{k}) on S_{±∞} (or S_{±[n]}) if wu^{−1} = (a, b) with a < k < b. The k-Bruhat order has a non-recursive characterization, needed below:

Proposition 2.1 ([3], Theorem A). Let u, w ∈ S_{±∞} and k ∈ ℤ. Then u ◁_{k} w if and only if

(1) a < k < b implies u(a) ≤ w(a) and u(b) ≥ w(b).

(2) If a < b, u(a) < u(b), and w(a) > w(b), then a < k < b.

For the remainder of this section, we will be concerned with the case k = 0.

Theorem 2.2. The 0-Bruhat order on B_{∞} is the order induced from the 0-Bruhat order on S_{±∞} by the inclusion B_{∞} ↪ S_{±∞}.

Proof. For u, w ∈ B_{∞}, it is straightforward to verify

u ⋖_{0} ut_{j} ⟺ u ◁·_{0} u(\overline{ȷ}, j)

and

u ⋖_{0} ut_{\overline{ı} j} ⟺ u ◁·_{0} u(\overline{ȷ}, i) ◁·_{0} u(\overline{ȷ}, i)(\overline{ı}, j).

Thus u <_{0} w ⇒ u ◁_{0} w, and so <_{0} is a suborder of ◁_{0}.

To show this suborder is induced, suppose that u ◁_{0} w with u, w ∈ B_{∞}, and argue by induction on l(w) − l(u). Suppose u ◁·_{0} v ◁_{0} w. If v = u(\overline{ȷ}, j) = ut_{j}, then v ∈ B_{n} and we are done by induction.

Suppose now that v = u(\overline{ȷ}, i) ∉ B_{n}. By the involution x ↦ ω_{0}xω_{0} of (S_{±∞}, ◁_{0}), ω_{0}vω_{0} = u(\overline{ı}, j) also satisfies u ◁·_{0} ω_{0}vω_{0} ◁_{0} w. The criteria of Proposition 2.1 show that either 0 < u(i) and 0 < u(j) so that u ◁·_{0} u(\overline{ȷ}, j) ◁_{0} w, or else u(j) · u(i) < 0 and so v ◁·_{0} v(\overline{ı}, j) ◁_{0} w. In the first case, u(\overline{ȷ}, j) = ut_{j} ∈ B_{n}, and in the second, v(\overline{ı}, j) = ut_{\overline{ı}, j} ∈ B_{n}, which completes the proof.

Remark 2.3. If u(\overline{ȷ}, i) ◁·_{0} u, then either ut_{ij} ⋖_{0} u or else both ut_{i} ⋖_{0} u and ut_{j} ⋖_{0} u.

Example 2.4. We illustrate Theorem 2.2 in Figure 3. There, the elements of B_{3} are boxed.

Figure 3. The intervals [\overline{3} 1 \overline{2} 2 \overline{1} 3, \overline{1} 2 3 \overline{3} \overline{2} 1]_{◁_{0}} and [2 \overline{1} 3, \overline{3} \overline{2} 1]_{0}.

This relation between the two partial orders (B_{∞}, <_{0}) and (S_{±∞}, ◁_{0}) makes many properties of (B_{∞}, <_{0}) easy corollaries of the analogous results for (S_{±∞}, ◁_{0}) (established in [3, 4]). We discuss these properties in the remainder of this section, omitting proofs.

The 0-Bruhat order has a non-recursive characterization:

Proposition 2.5. Let u, w ∈ B_{∞}. Then u ≤_{0} w if and only if

(1) 0 < i ⟹ u(i) ≥ w(i), and

(2) 0 < i < j and u(i) < u(j) ⟹ w(i) < w(j).

For P ⊂ {1, 2, … } = ℕ, let #P ∈ ℕ ∪ {∞} be the cardinality of P. Then the inclusion P ↪ ℕ induces compatible inclusions

[figure: ε_{P}; ε_{P}; B_{#P}; S_{±#P}; B_{∞}; S_{±∞}]

Shape equivalence is the equivalence relation on B_{∞} induced by u ∼ ε_{P} (u) for P ⊂ ℕ and u ∈ B_{#P}. Let [u, w]_{0} := {v | u ≤_{0} v ≤_{0} w} denote the interval in the 0-Bruhat order between u and w, a finite graded poset. A corollary of Theorems 2.2 and Theorem E(i) of [3] is the following fundamental result about the 0-Bruhat order on B_{∞}.

Theorem B(1) Suppose u, w, x, z ∈ B_{∞} with wu^{−1} shape equivalent to zx^{−1}. Then [u, w]_{0} ≃ [x, z]_{0}. If ε_{P} (wu^{−1}) = zx^{−1}, then this isomorphism is given by

[u, w]_{0} ∋ v ⟼ ε_{P} (vu^{−1})x ∈ [x, z]_{0}.

This property is not shared by the k-Bruhat order on B_{∞}, for any k > 0.

Example 2.6. Consider the following two intervals in the 1-Bruhat order on B_{4}:

Note that (3\overline{2}14)^{−1} · (3\overline{4}12) = (1\overline{2}43)^{−1} · (1\overline{4}23) = (\overline{2}, \overline{4})(2 , 4) and the two intervals are not isomorphic.

Remark 2.7. For any ζ ∈ B_{∞}, there is a u ∈ B_{∞} with u ≤_{0} ζu: Suppose {a ∈ ±ℕ | a > ζ(a)} = {a_{1}, …, a_{m}} with ζ(a_{1}) < ζ(a_{2}) < ⋯ < ζ(a_{m}). Further let {a_{m+1} < a_{m+2} < ⋯ } = ℕ \ {|a_{1}|, |a_{2}|, …, |a_{m}|}. If we define u(j) = a_{j}, then Proposition 2.5 implies that u ≤_{0} ζu. Note that if [m] = {a > 0 | ζ(a) ≠ a}, then ζu is a Grassmannian permutation.

By Theorem B(1), we may define a new partial order on B_{∞}, which we call the Lagrangian order: For η, ζ ∈ B_{∞}, set η ⪯ ζ if there is a u ∈ B_{∞} with u ≤_{0} ηu ≤_{0} ζu. By Remark 2.7, it has a unique minimal element e. This order is graded by the rank, L(ζ), where L(ζ) := ℓ(ζu) − ℓ(u) whenever u ≤_{0} ζu. These notions have definitions independent of ≤_{0}:

Definition-Theorem 2.8 (cf. Definition 3.2.2 [3]). Let η, ζ ∈ B_{∞}.

(1) Then η ⪯ ζ if and only if (i) a ∈ ±ℕ with a > η(a) ⟹ η(a) ≥ ζ(a), and (ii) a, b ∈ ±ℕ with a < b, a > ζ(a), b > ζ(b), and ζ(a) < ζ(b) ⟹ η(a) < η(b).

(2) L(ζ) = ∑_{a,0>ζ(a)} |ζ(a)| − #{(a, b) | 0 < a < b, a = ζ(a), a > ζ(b)}

− #{(a, b) | a < b, a > ζ(a), b > ζ(b), ζ(a) > ζ(b)} − ∑_{0>a>ζ(a)} |a|.

Proof. Let u be the permutation with u ≤_{0} ζu constructed from ζ in Remark 2.7. If u ≤_{0} ηu ≤_{0} ζu, then η satisfies the conditions in (i), and conversely.

For (ii), consider the difference ℓ(ζu) − ℓ(u). The length of ζu is the first sum, plus the number of inversions of the form 0 < i ≤ n < j with ζu(i) > ζu(j) = u(j). In the construction of u, each of these is also an inversion in u involving positions 0 < i ≤ n < j, and so are canceled in the difference. The second term counts the remaining inversions of this type in u, the third term counts the inversions with 0 < i < j ≤ n in u, and the fourth term is ∑_{i>0>u(i)} |u(i)|.

The Lagrangian order is the B_{∞}-counterpart of the Grassmann-Bruhat order ≺^{)} on S_{∞} of [3, 4]. This is defined as follows: Let η, ζ ∈ S_{∞}. Then η ≺^{)} ζ if and only if there is a u ∈ S_{∞} with u ◁_{0} ηu ◁_{0} ζu. The Grassmann-Bruhat order is ranked with ||η|| := l(ηu) − l(u) whenever u ◁_{0} ηu. Let s(ζ) count the sign changes {a > 0 | 0 > ζ(a)} in ζ. We have the following relation between these two orders.

Corollary 2.9.

(1) (B_{∞}, ≺) is an induced suborder of (S_{±∞}, ≺^{)} ).

(2) For ζ ∈ B_{∞}(⊂ S_{±∞}), we have L(ζ) = (||ζ|| + s(ζ))/2.

Proof. The first statement is a consequence of Theorem 2.2. For the second statement, consider any maximal chain in [e, ζ]_{≺} (in B_{∞}). By Theorem 2.2, this gives a maximal chain in [e, ζ]_{≺^{)}} (in S_{±∞}), where covers of the form η≺· t_{ab} η are replaced by η ≺·^{)} (a, b)η ≺·^{)} (a, b)(\overline{a}, \overline{b})η. Thus ||ζ|| = L(ζ) + τ, where τ counts the covers in that chain if the form η≺· t_{ab} η. Since only covers of the form η≺· t_{b}η contribute to s(ζ), we have ||ζ|| = 2L(ζ) − s(ζ).

Let η, ζ ∈ B_{∞}. If ζ · η = η · ζ with L(η · ζ) = L(η) + L(ζ), and neither of ζ or η is the identity, then η · ζ is the disjoint product of η and ζ. (In general L(η · ζ) ≤ L(η) + L(ζ).) If a permutation cannot be factored in this way, it is irreducible. Permutations ζ ∈ B_{∞} factor uniquely into irreducibles. This is described in terms of non-crossing partitions [23]: (A non-crossing partition of ±ℕ is a set partition such that if a < c < b < d with a, b in a part π and c, d in a part π′, then π = π′, as otherwise π, π′ are crossing.)

First, consider ζ as an element of S_{±∞}. Let Π be the finest non-crossing partition of ±ℕ which is refined by the partition given by the cycles of ζ. For each non-singleton part π of Π, let ζ_{π} be the product of the cycles of ζ which partition π. (These ζ_{π} are the irreducible factors of ζ, as an element of S_{±∞}.) Since ζ ∈ B_{∞}, for each such part π of Π, either π = \overline{π} or else one of π, \overline{π} consists solely of positive integers. In the first case, ζ_{π} is an irreducible factor of ζ (as an element of B_{∞}), and in the second, ζ_{π}ζ\overline{π} is an irreducible factor of ζ.

The main result concerning this disjointness is the following:

Theorem 2.10 (cf. Theorem G (i) [3]). Suppose ζ = ζ_{1} ⋯ ζ_{s} is the factorization of ζ ∈ B_{∞} into irreducibles. Then the map (η_{1}, …, η_{s}) ↦ η_{1} ⋯ η_{s} induces an isomorphism

[e, ζ_{1}]_{≺} × ⋯ × [e, ζ_{s}]_{≺} ⟶^{∼} [e, ζ]_{≺}.

This factorization into irreducibles suggests defining a type B non-crossing partition to be a non-crossing partition Π whose blocks π are either stable under ω_{0} (π = \overline{π}), or else π, \overline{π} are distinct, with one consisting solely of positive integers. These differ from the non-crossing partitions of type B introduced by Reiner [30], which form a graded lattice. Figure 4 shows the partitions of ±[2] defined here.

Figure 4. Non-crossing partitions in ±[2]

We summarize some properties of (B_{∞}, ≺).

Theorem 2.11 (cf. Theorem 3.2.3 of [3]).

(1) (B_{∞}, ≺) is a graded poset with minimal element e and rank function L(·).

(2) The map λ ↦ v(λ) exhibits the lattice of strict partitions as an induced suborder of (B_{∞}, ≺).

(3) If u ≤_{0} ζu, then η ↦ ηu induces an isomorphism [e, ζ]_{≺} ⟶^{∼} [u, ζu]_{0}.

(4) If η ⪯ ζ, then ξ ↦ ξη^{−1} induces an isomorphism [η, ζ]_{≺} ⟶^{∼} [e, ζη^{−1}]_{≺}.

(5) For every infinite set P ⊂ ℕ, the map ε_{P} : B_{∞} → B_{∞} is an injection of graded posets. Thus if η, ζ ∈ B_{∞} are shape equivalent, then [e, ζ]_{≺} ≃ [e, η]_{≺}.

(6) The map η ↦ ηζ^{−1} induces an order-reversing isomorphism between [e, ζ]_{≺} and [e, ζ^{−1}]_{≺}.

If ζ ∈ B_{∞} is factored into into disjoint cycles in S_{±∞}, the resulting cycles have one of two forms:

(a, b, …, c) or (a, b, …, c, \overline{a}, \overline{b}, …, \overline{c})

with |a|, |b|, …, |c| distinct. Furthermore, every cycle η = (a, b, …, c) of the first type is paired with another, \overline{η} := (\overline{a}, \overline{b}, …, \overline{c}), also of the first type. This motivates a ‘cycle notation’ for permutations ζ ∈ B_{∞}. Write ⟨a, b, …, c⟩ for the product (a, b, …, c) · (\overline{a}, \overline{b}, …, \overline{c}) and ⟨a, b, …, c] for cycles (a, b, …, c, \overline{a}, \overline{b}, …, \overline{c}) of the second type. Call either of these cycles in B_{∞}. In some examples and figures, the commas may be omitted. With this notation, Figure 5 shows the Lagrangian order on B_{3}. The thickened lines are between skew Grassmannian permutations v(λ)v(μ)^{−1} for μ ⊂ λ.

Figure 5. The Lagrangian order on B_{3}

2.2. A monoid for the Lagrangian order. The ‘Schubert vs. Schur’ structure constants b_{u λ}^{w} and c_{u λ}^{w} are related to the enumeration of (saturated) chains in the 0-Bruhat order on B_{∞}, and hence to the enumeration of chains in the Lagrangian order on B_{∞}. We develop the elementary theory of chains in these orders along the lines of [4].

A chain in either [u, ζu]_{0} or [e, ζ]_{≺} is a particular factorization of ζ into transpositions t_{b} and t_{a b}. We give an algorithm for finding a chain in [e, ζ]_{≺}. For this, set t_{\overline{b} b} = t_{b \overline{b}} = t_{b}.

Algorithm 2.12 (cf. Remark 3.1.2 [3]). input: A permutation ζ ∈ B_{∞}. output: Permutations ζ, ζ_{1}, …, ζ_{m} = e such that

e ≺· ζ_{m−1} ≺⋯ ≺· ζ_{1} ≺· ζ

is a saturated chain in the Lagrangian order.

Output ζ. While ζ ≠ e, do

1 Choose b ∈ ℕ maximal subject to b > ζ(b).

2 Choose a minimal subject to a ≤ ζ(b) < ζ(a).

3 ζ := ζt_{a b}, output ζ.

Before every execution of 3, ζt_{a b}≺· ζ. Moreover, this algorithm terminates in L(ζ) iterations and the reverse of the sequence produced is a chain in [e, ζ]_{≺}.

The Lagrangian order has a theory of reduced decompositions, analogous to the usual theory for the Coxeter group B_{∞} with respect to the weak order. (Also analogous to that for the Grassmann-Bruhat order on S_{∞} [4].) We express this in the context of monoids.

Define a monoid M with 0 and generators t_{a b}, t_{b} for integers 0 < a < b, one for each reflection of the Weyl group B_{∞}. To simplify the list of relations these satisfy, set t_{b b} := t_{b}. Also, if w ≡ u is a relation between words w and u in these generators, then w^{op} ≡ u^{op}, where w^{op} is the word w read backwards. That said, these generators are subject to the following relations:

(i)  t_{a c}t_{a}t_{a b} ≡ t_{a b}t_{b c}t_{b}  if a < b < c,
(ii) t_{b c}t_{c d}t_{a c} ≡ t_{b d}t_{a b}t_{b c}  if a < b < c < d
(iii)  t_{a b}t_{c d} ≡ t_{c d}t_{a b}  if a ≤ b < c ≤ d or a < c < d < b,
(iv)  t_{a c}t_{b} ≡ t_{b}t_{a c} ≡ 0  if a < b ≤ c or a = b = c,
(v)  t_{a c}t_{b d} ≡ t_{b d}t_{a c} ≡ 0  if a ≤ b < c ≤ d,
(vi) t_{b c}t_{a b}t_{b c} ≡ t_{a b}t_{b c}t_{a b} ≡ 0, if a < b < c.

(2)

These hold because M is a sub monoid of the monoid for the Grassmann-Bruhat order [4] in the same way the Lagrangian order is an induced suborder of the Grassmann-Bruhat order on S_{∞}.

The relation between M and the Lagrangian order on B_{∞} is obtained via a faithful representation of M as linear operators on ℚB_{∞}: Define linear operators t̂_{a b} and t̂_{b} on ℚB_{∞} by

t̂_{a b}.ζ := \{ \begin{matrix}t_{a b}ζ & if L(ζ) + 1 = |t_{a b}ζ| \\ 0 & otherwise. \end{matrix}
t̂_{b}.ζ := \{ \begin{matrix}t_{a}ζ & if L(ζ) + 1 = |t_{b}ζ| \\ 0 & otherwise. \end{matrix}

To simplify the following statement, let the index α represent either of b or a b.

Theorem 2.13 (cf. Theorem 1.1 [4]).

(1) The operators t̂_{α} satisfy the relations (1.1), and a composition of operators is characterized by its value at the identity. That is, t̂_{α′_{m}} ⋯ t̂_{α′_{1}} = t̂_{α_{n}} ⋯ t̂_{α_{1}} if and only if t̂_{α′_{m}} ⋯ t̂_{α′_{1}}e = t̂_{α_{n}} ⋯ t̂_{α_{1}}e.

(2) For x = t_{α_{n}} ⋯ t_{α_{2}}t_{α_{1}} ∈ M, the map x ↦ x̂ := t̂_{α_{n}} ⋯ t̂_{α_{2}}t̂_{α_{1}} is a faithful representation of M.

(3) The following map is a well defined bijection:

M ⟶ B_{∞} ∪ {0},
x ⟼ x̂e.

(4) For ζ, η ∈ B_{∞}, η ⪯ ζ if and only if there exists x ∈ M such that ζ = x̂η.

(5) The set R(ζ) = {x̂ : x̂e = ζ} corresponds to the set of all maximal chains in [e, ζ]_{≺}.

We call the elements of R(ζ) the ≺-reduced decompositions of ζ.

3. Isotropic flag manifolds and maximal Grassmannians

Let V denote either ℂ^{2n+1} equipped with non-degenerate symmetric bilinear form or ℂ^{2n} equipped with a non-degenerate alternating bilinear form. In the first case, V an odd orthogonal (vector) space, and in the second, a symplectic (vector) space. A linear subspace K of V is isotropic if the restriction of the form to K is identically zero. Isotropic subspaces have dimension at most n. An isotropic flag in V is a sequence E_{}of isotropic subspaces:

E_{}: E\overline{n} ⊂ E\overline{n−1} ⊂ ⋯ ⊂ E\overline{1},

where dim E\overline{ı} = n + 1 − i. Let K^{⊥} be the annihilator of a subset K of V. Given an isotropic flag E_{}in V, we obtain a canonical complete flag in V (also written E_{}) by defining E_{i} := E⊥ ; \overline{i+1} for i = 1, …, n, and in the odd orthogonal case, E_{0} := E⊥ ; \overline{1}. Henceforth, flags will always be complete, although we may only specify E\overline{n}, …, E\overline{1}.

The group G of linear transformations of V which preserve the given form acts transitively on the set of isotropic flags in V. Since the stabilizer of an isotropic flag is a Borel subgroup B of G, this exhibits the set of isotropic flags as the homogeneous space G/B. Here, G is either So_{2n+1}ℂ (odd orthogonal) or Sp_{2n}ℂ (symplectic). Similarly, G acts transitively on the set of maximal isotropic subspaces of V, exhibiting it as the homogeneous space G/P_{0}. Here P_{0} is the stabilizer of a maximal isotropic subspace, a maximal parabolic subgroup associated to the simple root of exceptional length. Let π : G/B ↠ G/P_{0} be the projection map.

The rational cohomology rings [9] of G/B for both the symplectic and odd-orthogonal flag manifolds are isomorphic to

ℚ[x_{1}, …, x_{n}]/⟨e_{i}(x_{1}^{2}, …, x_{n}^{2}), i = 1, …, n⟩,

where e_{i}(a_{1}, …, a_{n}) is the ith elementary symmetric polynomial in a_{1}, …, a_{n}. However, their integral cohomology rings differ [14]:

H^{∗}Sp_{2n}ℂ/B ≃ ℤ[x_{1}, …, x_{n}]/ ⟨e_{i}(x_{1}^{2}, …, x_{n}^{2})⟩ .
H^{∗}So_{2n+1}ℂ/B ≃ ℤ[x_{1}, …, x_{n}, c_{1}, …, c_{n}]/I,
I = ⟨e_{i}(x_{1}^{2}, …, x_{n}^{2}), 2c_{i} − e_{i}(x_{1}, …, x_{n}), c_{2i} = (−1)^{i}c_{i}^{2}⟩ .

These rings have another description in terms of Schubert classes.

3.1. Schubert varieties. Since an isotropic flag E_{}∈ G/B is also a complete flag in V, we have a canonical embedding G/B ↪ 𝔽ℓ(V), the manifold of complete flags in V. Similarly, there is an embedding G/P_{0} ↪ G_{n}(V), the Grassmannian of ndimensional subspaces of V. We use these maps to understand some structures of G/B.

Given w ∈ B_{n} and an isotropic (complete) flag E_{}∈ G/B, the Schubert variety Y_{w}E_{}of G/B is the collection of all flags F_{}∈ G/B satisfying

dim E_{i} ∩ F\overline{ȷ} ≥ #\{n ≥ l ≥ j | w(l) ≤ i\} (3)

for each j ∈ [n] and −n ≤ i ≤ n (i ≠ 0 in the symplectic case). This has codimension ℓ(w) in G/B. Also, Y_{w}E_{}⊂ Y_{u}E_{}if and only if u ≤ w in the Bruhat order. The Schubert cell Y_{u}^{∘}E_{}is the set of flags F_{}for which equality holds in (3). These are the flags in Y_{u}E_{}which are not in any sub-Schubert variety (Y_{w}E_{}with u ≤ w).

If now E_{}∈ 𝔽ℓ(V) and w ∈ S_{[\overline{n},n]} (S_{±[n]} in the symplectic case), then the Schubert variety X_{w}E_{}of 𝔽ℓ(V) is the collection of flags F_{}∈ 𝔽ℓ(V) satisfying 3) for all \overline{n} ≤ i, j ≤ n (i, j ≠ 0 in the symplectic case). Furthermore, if w ∈ B_{n} and E_{}∈ G/B, then

Y_{w}E_{}= G/B ⋂ X_{w}E_{}

and this is a scheme-theoretic equality [24].

The Schubert cells constitute a cellular decomposition of G/B. Thus Schubert classes, the cohomology classes Poincaré dual to the fundamental cycles of Schubert varieties, form ℤ-bases for these cohomology rings. Write 𝔅_{w} for the class [Y_{w}E_{}] in H^{∗}So_{2n+1}ℂ/B Poincaré dual to the fundamental cycle of Y_{w}E_{}of So_{2n+1}ℂ/B and ℭ_{w} for the corresponding class in H^{∗}Sp_{2n}ℂ/B. Since these are bases, there are integral structure constants b_{u v}^{w} and c_{u v}^{w} for u, w, v ∈ B_{n} defined by the identities:

𝔅_{u} · 𝔅_{v} = ∑_{w} b_{u v}^{w}𝔅_{w} and ℭ_{u} · ℭ_{v} = ∑_{w} c_{u v}^{w}ℭ_{w}.

Let s(w) count the number of sign changes in the permutation w. Then the isomorphism of rational rings is induced by the map [7]:

ℭ_{w} ⟼ 2^{s(w)}𝔅_{w}.

Thus

2^{s(u)+s(v)}b_{u v}^{w} = 2^{s(w)}c_{u v}^{w}. (4)

It suffices to establish identities and formulas for Sp_{2n}ℂ/B. We do this, because a crucial geometric result (Theorem 3.4(2)) does not hold for So_{2n+1}ℂ/B.

Two flags E_{}, E^{′} are opposite if dim(E_{i} ⋂ E′ ; \overline{ı}) = 1 for all i. In what follows, Y_{u} and Y′_{v}, will always denote Schubert varieties defined by fixed, but arbitrary, opposite isotropic flags. A consequence of Kleiman’s theorem on the transversality of a general translate [22], results in [13], and some combinatorics, is the following proposition.

Proposition 3.1. Let u, w ∈ B_{n}. Then Y_{u} ⋂ Y′_{ω_{0}w} ≠ 0 if and only if u ≤ w in the Bruhat order. If u ≤ w, then Y_{u}, Y′_{ω_{0}w} meet generically transversally, and the intersection cycle is irreducible of dimension ℓ(w) − ℓ(u).

The top-dimensional component of H^{∗}G/B is generated by the class of a point [pt] = 𝔅_{ω_{0}} or ℭ_{ω_{0}}. The map deg : H^{∗}G/B → ℤ selects the coefficient of [pt] in a cohomology class. The intersection pairing on H^{∗}G/B is the composition

β, γ ∈ H^{∗}G/B ⟼ deg(β · γ).

By Proposition 3.1, the product [Y_{u}] · [Y_{v}] is the cohomology class [Y_{u} ⋂ Y′_{v}]. In particular, when v = ω_{0}u, these intersections are single, reduced points, so that [Y_{u}] and [Y_{ω_{0}u}] are dual under the intersection pairing. Thus

c_{u v}^{w} = deg(ℭ_{u} · ℭ_{ω_{0}w} · ℭ_{v}),

which is also the number of points in the intersection

Y_{u} ⋂ Y′_{ω_{0}w} ⋂ Y′′_{v},

where Y′′_{v} is defined by a flag E^{′′} opposite to both E_{}and E^{′}(which define Y_{u} and Y′_{ω_{0}w}).

We derive a useful description of flags in Y_{u}^{∘} ⋂ Y′_{ω_{0}w}^{∘} when u ≤_{0} w. For S ⊂ ℂ^{m}, let ⟨S⟩ be the linear span of S.

Lemma 3.2. Suppose that u ≤_{0} w and E_{}, E^{′}are opposite isotropic flags in V. Then there are algebraic functions g\overline{ȷ} : Y_{u}^{∘}E_{}⋂ Y_{ω_{0}w}^{∘}E^{′} → V for 1 ≤ j ≤ n such that for

each flag F_{}∈ Y_{u}^{∘}E_{}⋂ Y_{ω_{0}w}^{∘}E^{′}and each 1 ≤ j ≤ n,

(1) F\overline{ȷ} = ⟨g\overline{n}(F_{}), …, g\overline{ȷ}(F_{})⟩, and

(2) g\overline{ȷ}(F_{}) ∈ E_{u(j)} ⋂ E′ ; \overline{w(j)}.

Proof. The representation of Schubert cells via parameterized matrices give V - valued functions f\overline{ȷ} defined on the Schubert cell Y_{u}^{∘}E_{}such that if F_{}is a flag in that cell, then F\overline{ȷ} = ⟨f\overline{n}(F_{}), …, f\overline{ȷ}(F_{})⟩, and f\overline{ȷ} ∈ E_{u(j)}.

Construct the functions g\overline{ȷ} inductively. First, set g\overline{n}(F_{}) := f\overline{n}(F_{}) for F_{}∈ Y_{u}^{∘}E_{}⋂ Y_{ω_{0}w}^{∘}E^{′}. Since F\overline{n} ⊂ E_{u(n)} ⋂ E′ ; \overline{w(n)}, conditions 1 and 2 are satisfied for g\overline{n}. Suppose we have constructed g\overline{ı} for n ≥ i > j. Let g\overline{ȷ}(F_{}) be the intersection of E′ ; \overline{w(j)} with the affine space

W_{j} := f\overline{ȷ}(F_{}) + ⟨g\overline{ı}(F_{}) | i > j and w(i) < w(j)⟩.

There is a unique point of intersection: Since F_{}∈ Y_{ω_{0}w}^{∘}E^{′},

dim E′ ; \overline{w(j)} ⋂ F\overline{ȷ} = #\{i | i ≥ j and w(i) ≥ w(j)\}.

Since u ≤_{0} w, if i > j and w(i) < w(j), then necessarily u(i) < u(j), by condition 2 of Proposition 2.5. Hence W_{j} ⊂ E_{u(j)} and so g\overline{ȷ}(F_{}) ∈ E_{u(j)} ⋂ E′ ; \overline{w(j)}.

Schubert varieties Υ_{λ} of G/P_{0} are indexed by strict partitions λ: decreasing sequences n ≥ λ_{1} > ⋯ > λ_{k} of positive integers. The projection map π : G/B ↠ G/P_{0} maps Schubert varieties to Schubert varieties, with πY_{u} = Υ_{λ}, where λ consists of the positive numbers among \{\overline{u(1)}, \overline{u(2)}, …, \overline{u(n)}\} arranged in decreasing order. For a strict partition λ, let v(λ) be the Grassmannian permutation whose (initial) negative values are \overline{λ_{1}} < \overline{λ_{2}} < ⋯ < \overline{λ_{k}}. Let λ^{c} be the decreasing sequence obtained from the integers in [n] which do not appear in λ. Then Y_{v(λ)} = π^{−1}Υ_{λ} and

π : Y_{ω_{0}v(λ)} ⟶ Υ_{λ^{c}}

is birational. One may see this by considering typical elements of their Schubert cells.

Set P_{λ} := [Υ_{λ}] in H^{∗}So_{2n+1}ℂ/P_{0} (equivalently P_{λ} := [Y_{v(λ)}] in H^{∗}So_{2n+1}ℂ/B) and let Q_{λ} be the corresponding class in the symplectic case. For 1 ≤ m ≤ n, the special Schubert variety Υ_{(m)} is the collection of all maximal isotropic subspaces which meet a fixed (n + 1 − m)-dimensional isotropic subspace. Let p_{m} (respectively q_{m}) denote the class P_{(m)} in either H^{∗}So_{2n+1}ℂ/B or H^{∗}So_{2n+1}ℂ/P_{0} (respectively the class Q_{(m)} in either H^{∗}Sp_{2n}ℂ/B or H^{∗}Sp_{2n}ℂ/P_{0}).

We are particularly interested in the constants b_{u λ}^{w} := b_{u v(λ)}^{w} and c_{u λ}^{w} := c_{u v(λ)}^{w} which give the structure of the cohomology of G/B as a module over the cohomology of G/P_{0}. Using the intersection pairing and the projection formula (cf. [18, 8.1.7]), we have

c_{u λ}^{w} = deg(ℭ_{u} · ℭ_{ω_{0}w} · π^{∗}(Q_{λ}))
= deg(π_{∗}(ℭ_{u} · ℭ_{ω_{0}w}) · Q_{λ}),

and a similar formula for b_{u λ}^{w}. Our main technique will be to find formulas for π_{∗}(ℭ_{u} · ℭ_{ω_{0}w}) by studying the effect of the map π on the cycle Y_{u} ∩ Y′_{ω_{0}w}.

To that end, define Y_{u}^{w} := π(Y_{u}∩Y′_{ω_{0}w}). These cycles Y_{u}^{w} are, like Schubert varieties, defined up to translation by G. In the theorems below, write Y_{u}^{w} = Y_{x}^{z} to mean that the cycles may be carried onto each other by an element of G. (The proofs are more explicit.)

The main result of Section 4 concerns these cycles.

Theorem 3.3. Let u, w ∈ B_{n} with u ≤_{0} w. Then

(1) The map π : Y_{u} ∩ Y′_{ω_{0}w} → Y_{u}^{w} has degree 1.

(2) If we have x, z ∈ B_{n} with z ≤_{0} z and wu^{−1} shape equivalent to zx^{−1}, then Y_{u}^{w} = Y_{x}^{z}.

This implies the identity of Theorem B(2). As a consequence of Theorem 3.3(2), define Y_{ζ} := Y_{u}^{ζu} for any ζ, u ∈ B_{n} with u ≤_{0} ζu.

These cycles satisfy more identities. Let W be a 2m-dimensional symplectic space. We study the manifolds of maximal isotropic subspaces of V, W, and V ⊕W, together with a map

Ξ : Sp_{2n}ℂ/P_{0} × Sp_{2m}ℂ/P_{0} ⟶ Sp_{2n+2m}ℂ/P_{0},

defined by (H, K) ↦ H ⊕ K. Also define ρ ∈ B_{n} by ρ(i) = i−1−n for 1 ≤ i ≤ n and γ ∈ B_{n} by γ(i) = i+1 for 1 ≤ i < n and γ(n) = 1.

Theorem 3.4.

(1) For any ζ ∈ B_{n}, Y_{ζ} = Y_{ζ^{−1}}.

(2) If η · ζ is a disjoint product in B_{n+m} with η′ ∈ B_{n} shape equivalent to η and ζ′ ∈ B_{m} shape equivalent to ζ, then

Ξ(Y_{η′} × Y_{ζ′}) = Y_{η·ζ}.

(3) For any ζ ∈ B_{n},

Y_{ζ} = Y_{ρζρ}.

(4) For any ζ ∈ B_{n} with a · ζ(a) > 0 for every a,

Y_{ζ} = Y_{γζγ^{−1}}.

The first statement follows from the observation that Y_{u} ⋂ Y_{ω_{0}ζu} = Y_{ω_{0}ζu} ⋂ Y_{ω_{0}ζ^{−1}(ω_{0}ζu)}. Since L(ζ) = dim Y_{ζ}, Theorem C(1) follows from part (3), by the projection formula. Similarly, Theorem C(2) follows from part (4). We remark that part (2) is true only for the symplectic case, while (1), (3), and (4) hold also for the odd orthogonal flag manifold. Statements (2), (3), and (4) are proven in Section 5.

4. Identities of structure constants

We establish Theorem 3.3 which implies Theorem B(2). As in Section 2, many results and methods are similar to those of [3] for analogous results about SL_{n}ℂ/B. Our discussions are therefore brief. The results here hold for both So_{2n+1}ℂ/B and Sp_{2n}ℂ/B, with nearly identical proofs. We only provide justification for Sp_{2n}ℂ/B.

Let H_{2} = ⟨h, \overline{h}⟩ ≃ ℂ^{2} be a symplectic space of dimension 2. Then the orthogonal direct sum V ⊕ H_{2} is a symplectic space of dimension 2n + 2. For each 1 ≤ p ≤ n + 1, define embeddings ψ_{p}, ψ\overline{p} : Sp_{2n}ℂ/B ↪ Sp_{2n+2}ℂ/B, the space of isotropic flags in V ⊕ H_{2}, by

(ψ_{p}E_{})_{j} = \{ \begin{matrix}E_{j+1} & j ≤ \overline{p} (< 0) \\ ⟨E_{j}, h⟩ & \overline{p} < j < 0 \end{matrix} .

Define ψ\overline{p} by replacing h with \overline{h} in the definition above. We find the effect of these maps on cohomology by determining the image of a Schubert variety under ψ_{p}.

First, define two maps between B_{n} and B_{n+1}. For every 1 ≤ p ≤ n + 1 and q ∈ ±[n + 1], define the injection ε_{p,q} : B_{n} ↪ B_{n+1} by:

ε_{p,q}(w)(j) = \{ \begin{matrix}w(j) & j < p and |w(j)| < |q| \\ w(j) − 1 & j < p and w(j) ≤ −|q| \\ w(j) + 1 & j < p and w(j) ≥ |q| \\ q & j = p \\ w(j − 1) & j > p and |w(j)| < |q| \\ w(j − 1) − 1 & j > p and w(j) ≤ −|q| \\ w(j − 1) + 1 & j > p and w(j) ≥ |q| \end{matrix} .

Let /_{p} : B_{n+1} → B_{n} be the left inverse of ε_{p,q}, defined by ε_{p,w(p)}(w/_{p}) = w. If we represent permutations as permutation matrices, then the effect of /_{p} on w ∈ B_{n+1} is to erase the pth and \overline{p}th columns and the w(p)th and \overline{w(p)}th rows. The effect of ε_{p,q} on w is to expand its permutation matrix with new pth, \overline{p}th columns and qth, \overline{q}th rows filled with zeroes, except for 1s at positions (q, p) and (\overline{q}, \overline{p}). For example:

\begin{matrix}ε_{3,\overline{2}}(\overline{2} 3 4 1) = \overline{3} 4 \overline{2} 5 1 \\ ε_{3,2}(\overline{2} 3 4 1) = \overline{3} 4 2 5 1 \end{matrix} and 4 \overline{1} 5 \overline{2} 3/_{4} = 3 \overline{1} 4 2.

These definitions imply the following proposition (cf. [31], Lemma 12).

Proposition 4.1. Let w ∈ B_{n}, 1 ≤ p, |q| ≤ n + 1, and E_{}any isotropic flag. Then

ψ_{p} Y_{w}E_{}⊂ Y_{ε_{p,q}(w)}ψ\overline{q}E_{}.

Recall that e is the identity permutation and Y_{e} is the flag manifold G/B.

Corollary 4.2. Let E_{}, E^{′} be opposite isotropic flags. Then for any q ∈ ±[n + 1], ψ_{q}E_{}, ψ\overline{q}E^{′} are opposite flags, and for any 1 ≤ p ≤ n + 1, ψ_{p} Y_{w}E_{}equals either of the following cycles

Y_{ε_{p,\overline{n+1}}(w)}ψ_{n+1}E_{}⋂ Y_{ε_{p,n+1}(e)}ψ\overline{n+1}E^{′} Y_{ε_{p,n+1}(w)}ψ\overline{n+1}E_{}⋂ Y_{ε_{p,\overline{n+1}}(e)}ψ_{n+1}E^{′}.

Proof. It is straightforward to check that the flags are opposite. Moreover, by Proposition 4.1, ψ_{p} Y_{w}E_{}is a subset of either intersection, as Y_{e} = Sp_{2n}ℂ/B. Since

ℓ(ε_{p,\overline{n+1}}(w)) = ℓ(w) + n + p, ℓ(ε_{p,n+1}(w)) = ℓ(w) + n + 1 − p,

and dim Sp_{2n}ℂ/B = n^{2}, Proposition 3.1 implies that all three cycles are irreducible with the same dimension, proving their equality.

Corollary 4.3. For any w ∈ B_{n} and 1 ≤ p ≤ n, we have

(ψ_{p})_{∗}ℭ_{w} = ℭ_{ε_{p,\overline{n+1}}(w)} · ℭ_{ε_{p,n+1}(e)} = ℭ_{ε_{p,n+1}(w)} · ℭ_{ε_{p,\overline{n+1}}(e)}.

Remark 4.4. As in Section 4.2 of [3], the Schubert classes ℭ_{ε_{p,n+1}(e)} and ℭ_{ε_{p,\overline{n+1}}(e)} are certain special Schubert classes from Grassmannian projections. Were the corresponding Pieri-type formulas known, we could deduce formulas for (ψ_{p})_{∗}ℭ_{w}. Only one of these classes is a special Schubert class from G/P_{0}:

𝔅_{ε_{1,\overline{n+1}}(e)} = p_{n+1} and ℭ_{ε_{1,\overline{n+1}}(e)} = q_{n+1}.

We deduce formulas for (ψ_{1})_{∗}𝔅_{w} and (ψ_{1})_{∗}ℭ_{w} from this.

Recall that H^{∗}Sp_{2n}ℂ/B is generated by x_{1}, …, x_{n}. These classes are Chern classes of certain line bundles on Sp_{2n}ℂ/B: Let E_{}→ Sp_{2n}ℂ/B be the flag of bundles whose fibre at E_{}is E_{}. Then x_{i} = −c_{1}(E\overline{i+1}/E\overline{ı}). Thus

ψ_{p}^{∗}(x_{i}) = \{ \begin{matrix}x_{i} & i < p \\ 0 & i = p \\ x_{i−1} & i > p \end{matrix} .

Theorem 4.5. Let v ∈ B_{n+1}. Then

(1) In H^{∗}So_{2n+1}ℂ/B, ψ_{1}^{∗}𝔅_{v} = ∑ θ(ε_{1,\overline{n+1}}(y)v^{−1}) 𝔅_{y}, the sum over all y ∈ B_{n} with ℓ(v) = ℓ(y).

(2) In H^{∗}Sp_{2n}ℂ/B, ψ_{1}^{∗}ℭ_{v} = ∑ χ(ε_{1,\overline{n+1}}(y)v^{−1}) ℭ_{y}, the sum over all y ∈ B_{n} with ℓ(v) = ℓ(y).

Proof. These are consequences of the projection formula and Corollary 4.3. For the first,

ψ_{1}^{∗}𝔅_{v} = ∑_{y∈B_{n}} deg(ψ_{1}^{∗}𝔅_{v} · 𝔅_{ω_{0}y})𝔅_{y}.

But

deg(ψ_{1}^{∗}𝔅_{v} · 𝔅_{ω_{0}y}) = deg(𝔅_{v} · (ψ_{1})_{∗}(𝔅_{ω_{0}y})) = deg(𝔅_{v} · 𝔅_{ε_{1,n+1}(ω_{0}y)} · q_{n+1})

Since ε_{1,n+1}(ω_{0}y) = ω_{0}(ε_{1,\overline{n+1}}(y)) and ℓ(ε_{1,\overline{n+1}}(y)) = ℓ(y) + n + 1, the result follows by the Pieri-type formula (Theorem D).

Lemma 4.6. Suppose u <_{0} w in B_{n+1} and u(p) = w(p) = q for some 1 ≤ p ≤ n + 1. Then

(1) u/_{p} <_{0} w/_{p} and ℓ(w) − ℓ(u) = ℓ(w/_{p}) − ℓ(u/_{p}).

(2) For any opposite isotropic flags E_{}, E^{′}in V,

ψ_{p} (Y_{u/_{p}}E_{}⋂ Y_{ω_{0}w/_{p}}E^{′}) = Y_{u}ψ\overline{q}E_{}⋂ Y_{ω_{0}w}ψ_{q}E^{′}.

Proof. Since u <_{0} w and u(p) = w(p), Proposition 2.5 implies that u/_{p} <_{0} w/_{p}. Moreover, wu^{−1} is shape equivalent to w/_{p}(u/_{p})^{−1}, so the first statement follows from Theorem B(1).

For (2), Proposition 4.1 gives the inclusion ⊂. By Corollary 4.2, ψ\overline{q}E_{}and ψ_{q}E^{′} are opposite flags. Thus, by Proposition 3.1 and (1), both sides are irreducible and have the same dimension, proving their equality.

Theorem 4.7. Suppose u <_{0} w in B_{n+1} and u(p) = w(p) = q for some 1 ≤ p ≤ n+1. Then for any strict partition λ, we have

b_{u λ}^{w} = b_{u/_{p} λ}^{w/_{p}}  and  c_{u λ}^{w} = c_{u/_{p} λ}^{w/_{p}}.

Proof. We first study the map Ψ : Sp_{2n}ℂ/P_{0} ↪ Sp_{2n+2}ℂ/P_{0}, defined by K ↦ ⟨K, h⟩. Here, Sp_{2n+2}ℂ/P_{0} is the Grassmannian of maximal isotropic subspaces of V ⊕ H_{2}. If E_{}, E^{′}are opposite isotropic flags in V, the analog of Corollary 4.2 is

Ψ(Υ_{λ}E_{}) = Υ_{λ}ψ_{n+1}E_{}⋂ Υ_{(n+1)}ψ\overline{n+1}E^{′},

where (n+1) is a decreasing sequence of length 1. We leave this to the reader. As this intersection is generically transverse, Ψ_{∗}Q_{λ} = Q_{λ} · q_{n+1}. Thus

Ψ^{∗}Q_{λ} = \{ \begin{matrix}Q_{λ} & λ_{1} < n + 1 \\ 0 & λ_{1} = n + 1 \end{matrix} .

To see this, note that Ψ^{∗}Q_{λ} = ∑_{μ} d_{λ}^{μ}Q_{μ}, where

d_{λ}^{μ} := deg((Ψ^{∗}Q_{λ}) · Q_{μ^{c}})
= deg(Q_{λ} · Ψ_{∗}(Q_{μ^{c}}))
= deg(Q_{λ} · q_{n+1} · Q_{μ^{c}}) = δ_{λ}^{μ},

the Kronecker delta, by the Pieri formula for isotropic Grassmannians [8]. Consider the commutative diagram:

[figure: Y_{u/_{p}}^{w/_{p}} = π(Y_{u/_{p}} ⋂ Y′_{ω_{0}w/_{p}}); Y_{u/_{p}} ⋂ Y′_{ω_{0}w/_{p}}; π(Y_{u} ⋂ Y′_{ω_{0}w}) = Y_{u}^{w}; Y_{u} ⋂ Y′_{ω_{0}w}; ψ_{p}; Ψ; π; π]

Thus [Y_{u}^{w}] = Ψ_{∗}[Y_{u/_{p}}^{w/_{p}}] and the maps π have the same degree δ as the horizontal maps are isomorphisms.

Let λ be a strict partition. Then c_{u λ}^{w} = δ · deg(Q_{λ} · [Y_{u}^{w}]) which is

δ · deg(Q_{λ} · Ψ_{∗}[Y_{u/_{p}}^{w/_{p}}]) = δ · deg(Ψ^{∗}(Q_{λ}) · [Y_{u/_{p}}^{w/_{p}}]) = δ · deg(Q_{λ} · [Y_{u/_{p}}^{w/_{p}}]) = c_{u/_{p} λ}^{w/_{p}}.

Lemma 4.8. Suppose u <_{0} w and x <_{0} z in B_{n} with wu^{−1} = zx^{−1} and u(i) ≠ w(i) for all i ∈ [n]. Then, if Y_{u}, Y_{x}, (respectively Y′_{ω_{0}w}, Y′_{ω_{0}z}) are defined with the same flags, there is commutative diagram

[figure: π(Y_{u} ⋂ Y′_{ω_{0}w}); Y_{u} ⋂ Y′_{ω_{0}w}; π(Y_{x} ⋂ Y′_{ω_{0}z}); Y_{x} ⋂ Y′_{ω_{0}z}; =; f; π; π]

where f is an isomorpism between Zariski open subsets of Y_{u} ⋂ Y′_{ω_{0}w} and Y_{x} ⋂ Y′_{ω_{0}z}, and the maps π have degree 1.

Proof of Theorem B(2). Let u, w, x, z ∈ B_{n} with wu^{−1} shape equivalent to zx^{−1} and λ a strict partition. If u(p) = w(p), then u/_{p} <_{0} w/_{p} with w/_{p}(u/_{p})^{−1} shape equivalent to wu^{−1}. Thus we may assume that the hypotheses of Lemma 4.8 hold, by Theorem 4.7. But then π_{∗}[Y_{u} ⋂ Y′_{ω_{0}w}] = π_{∗}[Y_{x} ⋂ Y′_{ω_{0}z}], hence π_{∗}(ℭ_{u} · ℭ_{ω_{0}w}) = π_{∗}(ℭ_{x} · ℭ_{ω_{0}z}), showing c_{u λ}^{w} = c_{x λ}^{z}.

Proof of Lemma 4.8. Let g\overline{ȷ} for 1 ≤ j ≤ n be the functions of Lemma 3.2, defined for all F_{}∈ Y_{u}^{∘} ⋂ Y′_{ω_{0}w}^{∘}. For such F_{}, let f(F_{}) be the flag whose \overline{ȷ}-th subspace is

⟨g_{u^{−1}x(\overline{n})}(F_{}), …, g_{u^{−1}x(\overline{ȷ})}(F_{})⟩.

Then f is an isomorphism between the intersections of Schubert cells, which are Zariski dense in the intersections of Schubert varieties, and the diagram is commutative. Since we may assume w is a Grassmannian permutation, and in this case, the map π : Y_{ω_{0}w} → π(Y_{ω_{0}w}) has degree 1, it follows that the maps π (which have the same degree) have degree 1.

5. More identities

5.1. Product decomposition. Suppose that W is a 2m-dimensional complex symplectic vector space and consider the map

Ξ : Sp_{2m}ℂ/P_{0} × Sp_{2n}ℂ/P_{0} ⟶ Sp_{2m+2n}ℂ/P_{0}

defined by

Ξ : (H, K) ⟼ H ⊕ K,

where H ⊂ W and K ⊂ V are maximal isotropic (Lagrangian) subspaces.

Theorem 5.1. Let η, ζ ∈ B_{n+m} with η · ζ a disjoint product and #supp(η) ≤ m, #supp(ζ) ≤ n. Then for any η′ ∈ B_{m} and ζ′ ∈ B_{n} with η ∼ η′ and ζ ∼ ζ′ (∼ is shape equivalence), there is an element g of Sp_{2m+2n}ℂ such that

Ξ(Y_{η′} × Y_{ζ′}) = g(Y_{η·ζ}).

Proof. This is a consequence of Lemma 5.2.1 of [3], the analogous fact for the classical flag manifold and Grassmannian. Restricting that result to the symplectic flag manifold and Grassmannian of maximal isotropic subspaces proves the theorem.

Remark. This does not hold for the odd orthogonal case. In fact, even the map Ξ cannot be defined: If V, W are odd-orthogonal spaces, then V ⊕ W is an even-dimensional space.

5.2. Some hidden symmetries. Define the permutation ρ ∈ B_{n} by ρ(i) = i−1−n. For example, in B_{4}, we have ρ = ⟨1, \overline{4}⟩⟨2, \overline{3}⟩, in the cycle notation of Section 2. In general, ρ = ⟨1, \overline{n}⟩⟨2, \overline{n−1}⟩ ⋯ ⟨⌊\frac{n+1}{2}⌋, ⌊\overline{\frac{n+2}{2}}⌋⟩. For a permutation ζ ∈ B_{n}, let ζ^{ρ} := ρζρ denote the conjugation of ζ by ρ.

A (saturated) chain in either the Lagrangian order or Lagrangian réseau gives a sequence a_{1}, …, a_{m} of labels. The peak set of that chain consists of those j ∈ {2, 3, …, m − 1} with a_{j−1} < a_{j} > a_{j+1}, its descent set of those i ∈ {1, 2, …, m − 1} with a_{i} > a_{i+1}, and its ascent set those i with a_{i} < a_{i+1}.

Theorem C(1). Let ζ ∈ B_{n}. Then L(ζ) = L(ζ^{ρ}) and if λ is a strict partition with |λ| = L(ζ), then

b_{λ}^{ζ} = b_{λ}^{ζ^{ρ}}  and  c_{λ}^{ζ} = c_{λ}^{ζ^{ρ}}.

Proof. The first statement follows from Corollary 2.9 as ||ζ|| = ||ζ^{ρ}|| in (S_{±[n]}, ≺^{)} ) and both ζ and ζ^{ρ} have the same number of sign changes. The last statement is a consequence of Lemma 5.4 below, which shows π_{∗}(ℭ_{u} · ℭ_{ω_{0}ζu}) = π_{∗}(ℭ_{x} · ℭ_{ω_{0}ζ^{ρ}x}) whenever u <_{0} ζu and x <_{0} ζ^{ρ}x.

Example 5.2. In B_{4}, let ζ = ⟨134⟩⟨2]. Then ζ^{ρ} = ⟨142⟩⟨3]. Consider the intervals [e, ζ]_{≺} and [e, ζ^{ρ}]_{≺} in the labeled Lagrangian réseau displayed in Figure 6. While they are not isomorphic, they have the same rank, the same number of maximal chains, 80, and the underlying orders each have 5 chains. Moreover, they each have 2 chains with peak set {3}, and one each with peak sets {2}, {4}, and {2, 4}. The réseaux have the same number of chains with fixed descent sets. The jth component of the following vector records the number of chains with descent set equal to the position of the 1’s in the binary representation of j−1:

(0, 2, 6, 4, 6, 12, 8, 2, 2, 8, 12, 6, 4, 6, 2, 0)

Definition 5.3. Let E_{}, E^{′}be opposite isotropic flags in V. Define a flag Ẽ_{}by:

Ẽ_{}: E_{1} ⋂ E′ ; \overline{1} ⊂ ⋯ ⊂ E_{n} ⋂ E′ ; \overline{1} ⊂ (E\overline{n} + E′ ; \overline{1}) ⊂ ⋯ ⊂ (E\overline{2} + E′ ; \overline{1}) ⊂ V.

Define Ẽ^{′}the same way, but with the roles of E_{}and E^{′}reversed. This gives opposite flags Ẽ_{}, Ẽ^{′}, and since (A ⋂ B)^{⊥} = (A^{⊥} + B^{⊥}), they are isotropic.

Figure 6. Conjugation by ρ on labeled intervals

Lemma 5.4. Suppose u, w, x, z ∈ B_{n} with ρu^{−1}wρ = x^{−1}z, and u(j) ≠ w(j) for 1 ≤ j ≤ n. Then, for any opposite isotropic flags E_{}, E^{′}in V, there is a commutative diagram

π(Y_{u}E_{}⋂ Y_{ω_{0}w}E^{′}) Y_{u}E_{}⋂ Y_{ω_{0}w}E^{′}  π(Y_{x}Ẽ_{}⋂ Y_{ω_{0}z}Ẽ^{′}) Y_{x}Ẽ_{}⋂ Y_{ω_{0}z}Ẽ^{′} = \overline{}f π  π [figure]  [figure]

with f an isomorpism of Zariski open subsets of Y_{u}E_{}⋂ Y_{ω_{0}w}E^{′}and Y_{x}Ẽ_{}⋂ Y_{ω_{0}z}Ẽ^{′}.

Proof. Let G_{}, G^{′}be oppsite (not necessarily isotropic) flags in V. Define G_{}^{+} to be

G_{}^{+} : G\overline{n−1} ⋂ G′_{n−1} ⊂ G\overline{n−2} ⋂ G′_{n−1} ⊂ ⋯ ⊂ G′_{n−1} ⊂ V.

Define G^{′}^{+} to be

G^{′}^{+} : G\overline{n} ⊂ (G\overline{n} + G′ ; \overline{n}) ⊂ ⋯ ⊂ (G\overline{n} + G′_{n−1}) ⊂ V.

For ζ ∈ S_{±[n]}, let ζ^{+} be the conjugation of ζ by the cycle (\overline{n}, …, \overline{1}, 1, …, n). In Section 5.3 of [3], the following proposition is proven:

Proposition 5.5. Let u, w, x, z ∈ S_{±[n]} with u ◁_{0} w, x ◁_{0} z, (u^{−1}w)^{+} = x^{−1}z, and w is a Grassmannian permutation with descent 0, (w(\overline{n}) < ⋯ < w(\overline{1}) and w(1) < ⋯ < w(n)). If π : 𝔽ℓV ↠ G_{n}(V) is the projection, then there is a commutative diagram:

π(X_{u}G_{}⋂ X_{ω_{0}w}G^{′}) X_{u}G_{}⋂ X_{ω_{0}w}G^{′}  π(X_{x}G_{}^{+} ⋂ X_{ω_{0}z}G^{′}^{+}) X_{x}G_{}^{+} ⋂ X_{ω_{0}z}G^{′}^{+} = \overline{}f π  π [figure]  [figure]

with f an isomorpism of Zariski open subsets of X_{u}G_{}⋂ X_{ω_{0}w}G^{′}and X_{x}G_{}^{+} ⋂ X_{ω_{0}z}G^{′}^{+}.

It suffices to prove Lemma 5.4 with w a Grassmannian permutation, by Lemma 4.8. Observe that (Ẽ_{}, Ẽ^{′}) is the result of n applications of the map (E_{}, E^{′}) ↦ (E_{}^{+}, E^{′}^{+}). Similarly, ρ = (\overline{n}, …, \overline{2}, \overline{1}, 1, …, n)^{n}. Thus, iterating Proposition 5.5 n times gives the commutative diagram in 𝔽ℓV and G_{n}V:

π(X_{u}E_{}⋂ X_{ω_{0}w}E^{′}) X_{u}E_{}⋂ X_{ω_{0}w}E^{′}  π(X_{x}Ẽ_{}⋂ X_{ω_{0}z}Ẽ^{′}) X_{x}Ẽ_{}⋂ X_{ω_{0}z}Ẽ^{′} = \overline{}f π  π [figure]  [figure]

Restricting this to the subset of isotropic flags gives the diagram of the lemma.

These same arguments prove the analog of Lemma 5.4 for So_{2n+1}ℂ/C.

5.3. More hidden symmetries. Until now, we have deduced identities in H^{∗}Sp_{2n}ℂ/B by rectricting constructions involving Schubert subvarieties of 𝔽ℓ(V) to those in Sp_{2n}ℂ/B via the embedding Sp_{2n}ℂ/B ↪ 𝔽ℓ(V). This is the geometric counterpart of the embedding B_{n} ↪ S_{±[n]} studied in Section 2. Here, we explore the geometry of the map ι : S_{n} ↪ B_{n}, where a permutation w ∈ S_{n} is extended to act on −[n] by w(−i) = −w(i). An immediate consequence of Definition-Theorem 2.8 and its analog for ≺^{)} is the following lemma.

Lemma 5.6. The map ι is an embedding of Bruhat orders (S_{n}, ◁) ↪ (B_{n}, ≤) and it respects the length functions in each order. Furthermore, ι(S_{n}) consists of those permutations ζ ∈ B_{n} with a · ζ(a) > 0 for all a.

Let L, L^{⊥} be complementary Lagrangian subspaces in V. The pairing (x, y) ∈ L ⊕ L^{⊥} ↦ β(x, y), where β is the alternating form, identifies them as linear duals. Given a subspace H of L, let H^{⊥} ⊂ L^{⊥} denote its annihilator in L^{⊥}. Then H + H^{⊥} is a Lagrangian subspace of V.

Let 𝔽ℓ(L) be the space of complete flags F_{}:= F_{1} ⊂ F_{2} ⊂ ⋯ ⊂ F_{n} = L in L. Note that here dim F_{i} = i. For each k = 0, 1, …, n, define an injective map

φ_{k} : 𝔽ℓ(L) ⟶ Sp_{2n}ℂ/B

by

(φ_{k}F_{})\overline{ȷ} = \{ \begin{matrix}F_{n+1−j} & j ≥ n − k + 1 \\ F_{k} + F_{k+j−1}^{⊥} & j ≤ n − k + 1 \end{matrix} . (5)

Then (φ_{k}F_{})\overline{1} = (F_{k} + F_{k}^{⊥}) is Lagrangian, showing that φ_{k}F_{}is an isotropic flag.

For w ∈ S_{n} the Schubert variety X_{w}E_{}of 𝔽ℓ(L) consists of those flags F_{}∈ 𝔽ℓ(L) satisfying

dim E_{a} ⋂ F_{b} = #{b ≥ l | w(l) + a ≥ n + 1}. (6)

We determine the image of Schubert varieties of 𝔽ℓ(L) under these maps φ_{k}. Define ϵ_{k} : S_{n} → B_{n} by

(ϵ_{k}w)(j) = \{ \begin{matrix}w(j + k) & 1 ≤ j ≤ n − k \\ \overline{w(n+1−j)} & n − k < j ≤ n \end{matrix} (7)

Note that ϵ_{0}w = ι(w).

Lemma 5.7. Let u, w ∈ S_{n} with u ◁_{k} w. Then ϵ_{k} induces an isomorphism of graded

posets [u, w]_{◁_{k}} ⟶^{∼} [u, w]_{0} and ι(wu^{−1}) = ϵ_{k}w(ϵ_{k}u)^{−1}.

Proof. A consequence of the definitions is that, for u, w ∈ S_{n},

u ◁_{k} w ⟺ ϵ_{k}u <_{0} ϵ_{k}w,

and ι(wu^{−1}) = ϵ_{k}w(ϵ_{k}u)^{−1}. The Lemma follows immediately from these observations.

Corollary 5.8. The map ι is an embedding of ranked orders ι : (S_{∞}, ≺^{)} ) ↪ (B_{∞}, ≺ ).

Let w^{∨} be defined by w^{∨}(j) = n+1−w(j). Then X_{w}E_{}and X_{w^{∨}}E^{′}are dual under the intersection pairing, where E_{}, E^{′}are opposite flags.

Lemma 5.9. With these definitions, φ_{k}X_{w}E_{}is a subset of either

Y_{ϵ_{k}w}φ_{n}E_{}or Y_{ω_{0}ϵ_{k}w^{∨}}φ_{0}E_{}.

Proof. Let F_{}∈ X_{w}E_{}. We show φ_{k}F_{}∈ Y_{ϵ_{k}w}φ_{n}E_{}, that is, for each −n ≤ i ≤ n and 1 ≤ j ≤ n (i ≠ 0),

dim (φ_{n}E_{})_{i} ⋂ (φ_{k}F_{})\overline{ȷ} ≥ #\{n ≥ l ≥ j | i ≥ ϵ_{k}w(l)\}. (8)

Suppose that j > n − k + 1. Then (φ_{k}F_{})\overline{ȷ} = F_{n+1−j} ⊂ L = (φ_{n}E_{})\overline{1}. If n ≥ l ≥ j, then (ϵ_{k}w)(l) = \overline{w(n+1−l)} < 0. Thus if i > 0, (8) holds as both sides equal n + 1 − j. Suppose i < 0. Then (φ_{n}E_{})_{i} = E_{n+1−\overline{ı}} and so the left side of 8) is

dim E_{n+1−\overline{ı}} ⋂ F_{n+1−j} ≥ #\{m ≤ n + 1 − j | w(m) + n + 1 − \overline{ı} ≥ n + 1\}
= #\{n ≥ l ≥ j | i ≥ \overline{w(n+1−l)} = ϵ_{k}w(l)\}.

Now suppose that j ≤ n − k + 1. Then (φ_{k}F_{})\overline{ȷ} = F_{k} + F_{k+j−1}^{⊥}. Thus the left side of (8) is

dim(φ_{n}E_{})_{i} ⋂ F_{k} + dim(φ_{n}E_{})_{i} ⋂ F_{k+j−1}^{⊥}. (9)

If i < 0, then (φ_{n}E_{})_{i} ⊂ L and only the first term of (9) contributes. By the previous paragraph, this is

dim(φ_{n}E_{})_{i} ⋂ F_{k} ≥ #{n ≥ l ≥ k | i ≥ ϵ_{k}w(l)}.

If k ≥ l, then ϵ_{k}w(l) > 0 > i, showing this equals the right side of (8).

If now i > 0, then (φ_{n}E_{})_{i} = L + E_{n−i}^{⊥}. Thus (9) is

k + dim E_{n−i}^{⊥} ⋂ F_{k+j−1}^{⊥} = k + n − dim(E_{n−1} + F_{k+j−1}).

But this is k + n − dim E_{n−1} − dim F_{k+j−1} + dim E_{n−1} ⋂ F_{k+j−1}, which is at least

i − j + 1 + #{k + j − 1 ≥ m ≥ 1 | w(m) + n − 1 ≥ n + 1}

= n − j + 1 − #{n ≥ m ≥ k + j | w(m) ≥ i + 1}
= k + #{n − k ≥ l ≥ j | w(l + k) ≤ i}
= k + #{n − k ≥ l ≥ j | ϵ_{k}w(l) ≤ i}.

This equals the right side of (8) since l > n − k implies ϵ_{k}w(l) < 0 < i.

Similar arguments show φ_{k}F_{}∈ Y_{ω_{0}ϵ_{k}w^{∨}}φ_{0}E_{}.

Corollary 5.10. Let u, w ∈ S_{n} with u ◁_{k} w and E_{}, E^{′}∈ 𝔽ℓ(L) be opposite flags. Then φ_{n}E_{}, φ_{0}E^{′}are opposite isotropic flags, and

φ_{k} (X_{u}E_{}⋂ X_{w^{∨}}E^{′}) = Y_{ϵ_{k}u}φ_{n}E_{}⋂ Y_{ω_{0}ϵ_{k}w}φ_{0}E^{′}.

Proof. Lemma 5.9 gives the inclusion ⊂ and it is easy to see that φ_{n}E_{}, φ_{0}E^{′}are opposite. By Lemma 5.7, both sides have the same dimension, proving equality.

For each k = 1, 2, …, n, let π_{k} : 𝔽ℓ(L) → G_{k}(L) be the projection induced by E_{}↦ E_{k}. As in Lemma 4.8, if u ◁_{k} w in S_{n} and E_{}, E^{′} are opposite flags in 𝔽ℓ(L), then the intersection X_{u}E_{}⋂ X_{w^{∨}}E^{′} is mapped birationally onto its image π_{k}(X_{u}E_{}⋂ X_{w^{∨}}E^{′}) in G_{k}(L). Furthermore, the image cycle depends only upon η := wu^{−1}. Denote it by X_{η}.

Define Φ_{k} : G_{k}(L) → Sp_{2n}ℂ/B by H ↦ (H + H^{⊥}). Then Φ_{k} ∘ π_{k} = π ∘ φ_{k} and we have the following corollary.

Corollary 5.11. For any η ∈ S_{n}, Y_{ι(η)} = Φ_{k}(X_{η}), where k = #{a | a < η(a)}.

Recall that γ := ι(1, 2, …, n) ∈ B_{n}.

Theorem C(2). Let ζ ∈ ι(S_{n}). Then L(ζ) = L(ζ^{γ}) and if λ is a strict partition with |λ| = L(ζ), then

b_{λ}^{ζ} = b_{λ}^{ζ^{γ}}  and  c_{λ}^{ζ} = c_{λ}^{ζ^{γ}}.

Proof. The first statement follows from Lemma 5.6 as ||η|| = ||η^{γ}|| for η ∈ S_{n}. The last statement is a consequence of the identity X_{η} = X_{η^{γ}} (Proposition 5.5) and Corollary 5.11.

Example 5.12. Let η = (1, 2, 4, 3). Then ζ = ι(η) = ⟨1, 2, 4, 3⟩ and ζ^{γ} = ⟨1, 4, 2, 3⟩. The labeled intervals [e, η]_{≺^{)}} and [e, ζ]_{≺} are isomorphic. Consider the intervals [e, ζ]_{≺} and [e, ζ^{γ}]_{≺} in the labeled Lagrangian réseau displayed in Figure 7. While they are not isomorphic, they have the same rank, the same number of maximal chains, 16, and the underlying orders each have 2 maximal chains. Moreover, they each have a peakless chain and one with peak set {2}. The réseaux each have 2 increasing chains, 2 decreasing chains, 6 with descent set {1}, and 6 with descent set {2}.

Figure 7. Conjugation by γ on labeled intervals

6. Minimal permutations and labeled réseaux

6.1. Minimal permutations. For a cycle ζ ∈ B_{∞}, let δ(ζ) = 1 if ζ has the form ι(η) for η ∈ S_{∞} and δ(ζ) = 0 otherwise. Note that δ(ζ) = 1 if and only if a > 0 implies ζ(a) > 0.

Lemma 6.1. Let ζ ∈ B_{∞} be a cycle. Then L(ζ) ≥ #supp(ζ) − δ(ζ).

Recall that s(ζ) counts the number of sign changes in ζ. Proof. A saturated chain in [e, ζ]_{≺^{)}} (in S_{±∞}) gives a factorization of ζ into transpositions. If ζ consists of two cycles in S_{±∞}, then ||ζ|| ≥ 2(#supp(ζ) − 1) and by Corollary 2.9(2),

L(ζ) ≥ #supp(ζ) − 1 + s(ζ) ≥ #supp(ζ) − δ(ζ),

with equality only if s(ζ) = 0, that is, only if δ(ζ) = 1.

Similarly, if ζ is a single cycle in S_{±∞}, then ||ζ|| ≥ 2#supp(ζ) − 1. Since δ(ζ) = 0 and s(ζ) ≥ 1, Corollary 2.9(2) gives

L(ζ) ≥ #supp(ζ) − δ(ζ).

Corollary 6.2. If ζ ∈ B_{∞} is irreducible and L(ζ) = #supp(ζ) − δ(ζ), then ζ is a single cycle.

Proof. Recall that if η, ξ ∈ B_{∞} have disjoint supports, then L(η · ξ) ≥ L(η) + L(ξ), with equality only when η · ξ is a disjoint product. Thus, by Lemma 6.1,

L(ζ) ≥ #supp(ζ) − δ(ζ),

with equality only when ζ is a single cycle.

Definition 6.3. A minimal cycle is a cycle ζ ∈ B_{∞} for which L(ζ) = #supp(ζ)−δ(ζ). For minimal cycles, s(ζ) + δ(ζ) = 1. A permutation ζ ∈ B_{∞} is minimal if each of its irreducible factors are minimal cycles.

Corollary 6.4. If η, ζ ∈ B_{∞} with η ≺ ζ and ζ is minimal, then so is η

Proof. By Theorem 2.10, we may assume ζ is irreducible. Then ζ is a single cycle and the result follows by induction on L(ζ), similar to the proof of Lemma 6.1.

Corollary 6.5. Let ζ ∈ B_{∞}. If ζ = ζ_{1} ⋯ ζ_{s} is the factorization of ζ into irreducibles, then

L(ζ) ≥ #supp(ζ) − ∑_{i} δ(ζ_{i}),

with equality only if ζ is minimal.

Lemma 6.6. If ζ ∈ B_{∞} is a minimal cycle with δ(ζ) = 0, then there is a unique a ∈ supp(ζ) with a > 0 > ζ(a) =: \overline{α}. Furthermore,

\begin{matrix}1 ≺· t_{α} ⪯ ζ & if a ≤ α \\ 1 ⪯ t_{α}ζ ≺· ζ & if a ≥ α. \end{matrix}

Proof. Let ζ ∈ B_{∞} be a minimal cycle with δ(ζ) = 0. Then s(ζ) = 1, so there is a unique a > 0 with \overline{α} := ζ(a). We prove the lemma when a > α: If a = α, then ζ = t_{a}, as ζ is irreducible and if a < α, then replacing ζ by ζ^{−1}, reduces to the case a > α.

Suppose a > α. We claim that if b > a, then ζ(b) > α. The lemma follows from this claim. Indeed, then condition (ii) of Definition 2.8(1) is satisfied, and hence t_{α}ζ ≺ ζ. Since a < α, we have supp(t_{α}ζ) = supp(ζ). As δ(t_{α}ζ) = 1, it follows that L(t_{α}ζ) ≥ #supp(ζ) − 1 = L(ζ) − 1 and thus t_{α}ζ≺· ζ.

Let ζ be a permutation for which the claim does not hold with #supp(ζ) minimal. By Algorithm 2.12, η := ζt_{y x}≺· ζ, where x is maximal in supp(ζ) and y is minimal subject to y ≤ ζ(x) < ζ(y) ≤ x. Note that y ≠ a. Since a < x, δ(η) = 0. As ζ is minimal and η≺· ζ, either η is irreducible and supp(η) ⊊ supp(ζ) or else η is the disjoint product of two minimal cycles with x in the support of one and |y| in the support of the other.

If y < 0, then y = \overline{a} as ζ(y) > 0. Then η(a) = \overline{ζ(x)} and η(x) = α, so supp(η) ⊂ supp(ζ) and so η is irreducible with x in the support of one component and a in the other. But x > a > α = η(x) > \overline{ζ(x)} = η(a), contradicting disjointness.

Suppose now that y > 0. Then ζ(x) > α, for otherwise y = \overline{a} < 0. Thus x > b. Since b > a > α > η(b) with η(a) = \overline{α}, the component η′ of η whose support contains a also contains b and supp(η′) ⊊ supp(ζ) with δ(η′) = 0, contradicting the minimality of #supp(ζ).

6.2. The Grassmann-Bruhat order on S_{∞}. We develop some additional combinatorics for the symmetric group S_{∞}. For k ∈ ℕ, the k-Bruhat order (◁_{k}) on S_{∞} is the analog of the k-Bruhat order on B_{±∞}. The interval [u, w]_{k} in the k-Bruhat order depends only upon wu^{−1}, so we define the Grassmann-Bruhat order on S_{∞} (the Lagrangian order on B_{∞} is its analog) by η ≺^{)} ζ if there is a k ∈ ℕ and u ∈ S_{∞} with u ◁_{k} ηu ◁_{k} ζu. This order is ranked by ||ζ|| = l(ζu) − l(u) for u ◁_{k} ζu, and it has an independent description:

Definition 3.2.2 of [3] Let η, ζ ∈ S_{∞}. Then η ≺^{)} ζ if and only if

(1) a < η(a) ⇒ η(a) ≤ ζ(a)

(2) a > η(a) ⇒ η(a) ≥ ζ(a)

(3) If a < ζ(a) and b < ζ(b) (respectively a > ζ(a) and b > ζ(b)) with a < b and ζ(a) < ζ(b), then η(a) < η(b).

Covers η ≺·^{)} ζ correspond to transpositions (α, β) = ζη^{−1} and we construct a labeled Hasse diagram for (S_{∞}, ≺^{)} ), labeling such a cover with the greater of α, β. By Theorem 3.2.3 of [3], the map η ↦ ζη^{−1} induces an order-reversing isomorphism between [e, ζ]_{≺^{)}} and [e, ζ^{−1}]_{≺^{)}} , preserving the edge labels. Also, if P = {p_{1}, p_{2}, … } ⊂ ℕ and ε_{P} : S_{P} ↪ S_{∞} is the map induced by the inclusion P ↪ ℕ (these maps induce shape equivalence), then ε_{P} induces an isomorphism [e, ζ]_{≺^{)}} ⟶^{∼} [e, ε_{P} (ζ)]_{≺^{)}} , preserving the relative order of the edge labels. Specifically, an edge label i of [e, ζ]_{≺^{)}} is mapped to the label p_{i} of [e, ε_{P} (ζ)]_{≺^{)}} . Lastly, we remark that Algorithm 2.12 restricted to S_{∞}, and with t_{a b} replaced by the transposition (a, b), gives a chain in the ≺^{)} -order on S_{∞} from e to ζ.

Lemma 6.7. Let ζ ∈ S_{∞} and suppose that x is maximal subject to x ≠ ζ(x). Then, for any α

(1) (α, x)ζ ≺^{)} ζ ⟹ ζ^{−1}(x) ≤ ζ(x) = α.

(2) (α, x) ≺^{)} ζ ⟹ ζ^{−1}(x) ≥ ζ(x) = α.

Proof. For 1, let η := (α, x)ζ ≺^{)} ζ and set a = ζ^{−1}(x) and b = ζ^{−1}(α). Note that a ≠ b and η(b) = x. We claim that b = x and a < α, which will establish 1.

Suppose b ≠ x = η(b). Then, by the maximality of x, b < η(b) and so the definition of ≺^{)} implies η(b) ≤ ζ(b) = α. Since α < x, this implies x < x, a contradiction. Suppose now that a > α = η(a). By the definition of ≺^{)} , this implies that η(a) ≥ ζ(a) = x, and so a > x, contradicting the maximality of x.

The second assertion follows from the first by applying the anti-isomorphism η ↦ ηζ^{−1} between [e, ζ]_{≺^{)}} and [e, ζ^{−1}]_{≺^{)}} ,

e ≺^{)} (α, x) ≺^{)} ζ ⟺ (α, x)ζ^{−1} ≺^{)} ζ^{−1}.

A cycle ζ ∈ S_{∞} is minimal if ||ζ|| + 1 = #supp(ζ). A permutation is minimal if it is the disjoint product of minimal cycles. A maximal chain in an interval [e, ζ]_{≺^{)}} is peakless if we do not have a_{i−1} < a_{i} > a_{i+1} for any i = 2, …, ||ζ||−1, where a_{1}, …, a_{||ζ||} is the sequence of labels in that chain.

Lemma 6.8. Suppose ζ ∈ S_{∞} is a minimal cycle. Then there is a unique peakless chain in the labeled interval [e, ζ]_{≺^{)}} . If β is the smallest label in such a chain, then the transposition of that cover is (α, β) where α < β are the two smallest elements of supp(ζ).

Proof. We argue by induction on ||ζ||, which we assume is at least 2, as the case ||ζ|| = 1 is immediate. Replacing ζ by a shape equivalent permutation if necessary, we may assume that supp(ζ) = [n], so that ||ζ|| = m = n − 1.

Replacing ζ by ζ^{−1} would only reverse such a chain, so we may assume that a := ζ^{−1}(n) < b := ζ(n). We claim that (b, n)ζ = ζ(a, n) ≺·^{)} ζ. Given this, the conclusion of the lemma follows. Indeed, let η := (b, n)ζ. Since η(n) = n, this is an irreducible minimal permutation in S_{n−1}. By the inductive hypothesis, [e, η]_{≺^{)}} has a unique chain with labels β_{1} > ⋯ > β_{k} < ⋯ < β_{n−2}, and each β_{i} < n. The unique extension of this to a chain in [e, ζ]_{≺^{)}} has β_{n−2} < β_{n−1} = n. This is the unique such chain in [e, ζ]_{≺^{)}} as η ≺·^{)} ζ is the unique terminal cover in [e, ζ]_{≺^{)}} with edge label n, by Lemma 6.7. Also note that unless n = 2, 1 ≤ a < b, which proves the second part of 1.

By Algorithm 2.12, if y is chosen minimal so that y ≤ ζ(n) = b < ζ(y), then ζ(y, n) ≺·^{)} ζ. We show that y = a, which will establish the claim and complete the proof.

Suppose y ≠ a. Since a < b < n = ζ(a), the minimality of y implies that y < a. But then ζ(y, n) consists of two cycles η and η′ and we have η(a) = n and η′(y) = b. Since y < a < b < n, these cycles are not disjoint, so we have

n − 2 = ||η · η′|| > ||η|| + ||η′||
≥ #supp(η) − 1 + #supp(η′) − 1 = n − 2,

a contradiction.

6.3. The labeled Lagrangian order. The labeled Lagrangian and 0-Bruhat orders on B_{∞} are obtained from the Hasse diagrams of the underlying orders by labeling each cover with the integer β, where that cover is either ζ≺· t_{β}ζ (u≺· t_{β}u) or ζ≺· t_{α β}ζ (u≺· t_{α β}u). Recall that the map ι : S_{∞} → B_{∞} maps the labeled Grassmann-Bruhat order on S_{∞} isomorphically onto its image in the labeled Lagrangian order, preserving edge labels. The Pieri-type formula for So_{2n+1}ℂ/B has two formulations (Theorems A and D), which we relate here. We say that a chain in [e, ζ]_{≺} is peakless if in its sequence β_{1}, …, β_{m} of labels, we do not have β_{i−1} < β_{i} > β_{i+1}, for any i = 2, …, m − 1.

Lemma 6.9. Let ζ ∈ B_{∞} be a minimal cycle. Then there is a unique peakless chain in the labeled interval [e, ζ]_{≺}. If δ(ζ) = 0, then the minimal label corresponds to the cover whose reflection is of the form t_{a}.

Proof. If δ(ζ) = 1 this is an immediate consequence of Lemma 6.8. Suppose that δ(ζ) = 0. Replacing ζ by a shape equivalent permutation if necessary, we may assume that supp(ζ) = [n] and n > 1. Replacing ζ by ζ^{−1} if necessary, we may assume that a := ζ^{−1}(n) < b := ζ(n). As in the proof of Lemma 6.8, (b, n)ζ ≺·^{)} ζ in the Grassmann-Bruhat order on S_{±∞}. By Remark 2.3, either t_{b n}ζ≺· ζ or else we have both t_{b}ζ≺· ζ and t_{n}ζ≺· ζ. The second case implies s(ζ) > 1, contradicting the minimality of ζ. Thus η := t_{b n}ζ≺· ζ.

Then η is a minimal cycle with δ(η) = 0 and supp(η) = [n−1]. Appending the cover η ⟶^{n} ζ to the unique peakless chain in [e, η]_{≺} gives a peakless chain in [e, ζ]_{≺}.

Moreover, η is the unique permutation with η≺· ζ and η ⟶^{n} ζ, showing the uniqueness of this chain.

For any ζ ∈ B_{∞}, let Π(ζ) be the number of peakless chains in [e, ζ]_{≺}.

Lemma 6.10. If η, ζ ∈ B_{∞} are disjoint, then

Π(η · ζ) = 2 Π(η) · Π(ζ).

Proof. For ξ ∈ B_{∞}, let W (ξ) be the multiset of words formed from labels of maximal chains in [e, ξ]_{≺}. The alphabet of these words is a subset of supp(ξ). Thus W (η) and W (ζ) have disjoint alphabets. Note that W (η · ζ) consists of all pairs of words in W (η) × W (ζ). The lemma follows from Lemma 6.11, a combinatorial result concerning peakless words and shuffles.

For a set A of words in an ordered alphabet A, let peak(A) be the subset of peakless words from A. Suppose that A′ is another set of words with a different alphabet A′ and fix some total order on the disjoint union A ∐ A′ which extends the given orders on each of A, A′. Let sh(A, A′) be all shuffles of pairs of words in A × A′.

Lemma 6.11. The natural restriction map sh(A, A′) ↠ A×A′ induces a 2 to 1 map

peak(sh(A, A′)) −↠ peak(A) × peak(A′).

Proof. It is clear that the restriction map takes a peakless word in sh(A, A′) to a pair of peakless words in A × A′. Given a pair of peakless words (ω, ω′) ∈ A × A′, there are exactly two shuffles of ω, ω′ which are peakless: Suppose the minimal letter a in ω is greater than the minimal letter in ω′. Then these two shuffles differ only in their subwords consisting of a and u′, where u′ is that subword of ω′ consisting of all letters less than a. Then u′ is a segment of ω′, as ω′ is peakless. The two subwords of peakless shuffles are a.u′ and u′.a.

Lemma 6.12. Let ζ ∈ B_{∞} and suppose there is a peakless chain in [e, ζ]_{≺}. Then ζ is minimal.

Proof. Suppose by way of contradiction that ζ ∈ B_{∞} is irreducible and not minimal, but Π(ζ) ≠ 0. We may further assume that among all such permutations, ζ has minimal rank, and that supp(ζ) = [n]. Let β_{1} > ⋯ > β_{k} < ⋯ < β_{m} be the labels in a peakless chain in [e, ζ]_{≺}. Replacing ζ by ζ^{−1} if necessary (which merely reverses the chain), we may assume that β_{m} = n and so β_{1} ≠ n, by Lemma 6.7 and Theorem 2.2. Let η be the penultimate member of this chain. Then Π(η) ≠ 0, as the initial segment of this chain gives a peakless chain in [e, η]_{≺}. Thus η is a minimal permutation, by our assumption on ζ, and so

|η| ≤ #supp(η) − δ(η) ≤ n − δ(ζ) < L(ζ) = |η| + 1,

as ζ is not minimal and η≺· ζ so δ(η) ≥ δ(ζ). Therefore the weak inequalities must be equalities, so that supp(η) = [n]. Since β_{1} > ⋯ > β_{k} < ⋯ < β_{m−1} are the labels of a chain in [e, η]_{≺} and β_{m−1} < n, we must have β_{1} = n, as supp(η) = [n]. But this contradicts our earlier observation about β_{1}.

We relate the two formulations of the Pieri formula in the odd-orthogonal case. For ζ ∈ B_{∞}, define

θ(ζ) = \{ \begin{matrix}2^{#\{irreducible factors of ζ\}−1} & if ζ is minimal \\ 0 & otherwise \end{matrix}

Corollary 6.13. For ζ ∈ B_{∞}, Π(ζ) = θ(ζ).

Proof. This is clear if ζ is minimal as both Π(ζ) and θ(ζ) satisfy the same recursion, by Lemmas 6.9 and 6.10.

6.4. The Lagrangian réseau. The enumerative significance of the structure constants c_{u v}^{w} is best expressed in terms of maximal chains in certain directed multigraphs associated to intervals in the Bruhat order, which we call labeled réseaux. A cover η≺· ζ in the Lagrangian order corresponds to a reflection ηζ^{−1}, which is either of the form t_{a b} or of the form t_{a}. The Lagrangian réseau on B_{∞} is the labeled directed multigraph where a cover η≺· ζ in the Lagrangian order with ηζ^{−1} = t_{a} is given a single edge η ⟶^{a} ζ and a cover with ηζ^{−1} = t_{a b} is given two edges η ⟶^{\overline{a}} ζ and η ⟶^{b} ζ. We obtain the labeled Lagrangian order from this réseau by erasing those edges whose negative labels.

In the Grassmann-Bruhat order on S_{∞}, there are two conventions for labeling a cover η ≺·^{)} ζ: This cover gives a transposition (α, β) := ηζ^{−1} with α < β, and we may choose either α or β. For want of a better term, we call the consistent choice of α the lower convention, and the consistent choice of β the upper convention. We make use of the following fact.

Proposition 6.14. Let η ∈ S_{∞}. If there is a chain in [e, η]_{≺^{)}} with decreasing labels in the lower convention, then there is a chain in [e, η]_{≺^{)}} with decreasing labels in the upper convention, and these chains are unique. The same is true for chains with increasing labels, and in either case η is minimal.

A chain with increasing labels is an increasing chain and one with decreasing labels is a decreasing chain.

Lemma 6.15. Let ζ ∈ B_{∞} be a minimal cycle. Then the réseau [e, ζ]_{≺} has an increasing chain. If δ(ζ) = 1, then there are at least 2 increasing chains.

Proof. Consider the peakless chain in the labeled order [e, ζ]_{≺}:

e ⟶^{α_{1}} ζ_{1} ⟶^{α_{2}} ⋯ ⟶^{α_{m}} ζ_{m} = ζ (10)

Let α_{k} be the minimal label in this chain. Then ζ_{k−1} = ι(η_{k−1}) for some η_{k−1} ∈ S_{∞}. To see this, if δ(ζ) = 0, then by Lemma 6.9, the label α_{k} corresponds to the only cover whose reflection is not in ι(S_{∞}), and so δ(ζ_{k−1}) = 1.

The pullback of the initial segment of this chain to [e, η_{k−1}]_{≺^{)}} gives a decreasing chain (with labels α_{1}, …, α_{k−1}) in the upper convention. Consider the unique decreasing chain

e ⟶^{β_{1}} η_{1} ⟶^{β_{2}} ⋯ ⟶^{β_{k−1}} η_{k−1}

in the lower convention. Then

e ⟶^{\overline{β_{1}}} ι(η_{1}) ⟶^{\overline{β_{2}}} ⋯ ⟶^{\overline{β_{k−1}}} ι(η_{k−1}) = ζ_{k−1}

is an increasing chain in the réseau [e, ζ_{k−1}]_{≺}. Concatenating the end of the peakless chain (10) onto this gives an increasing chain

e ⟶^{\overline{β_{1}}} ι(η_{1}) ⟶^{\overline{β_{2}}} ⋯ ⟶^{\overline{β_{k−1}}} ι(η_{k−1}) ⟶^{α_{k}} ⋯ ⟶^{α_{m}} ζ

in the réseau e, ζ]_{≺}.

Suppose δ(ζ) = 1 and consider the middle portion of this increasing chain:

ι(η_{k−2}) ⟶^{\overline{β_{k−1}}} ι(η_{k−1}) = ζ_{k−1} ⟶^{α_{k}} ζ_{k}.

Let \overline{b} be the label of the other edge between ζ_{k−1} and ζ_{k}. Then we claim that \overline{β_{k−1}} < \overline{b}

so that replacing ζ_{k−1} ⟶^{α_{k}} ζ_{k} by ζ_{k−1} ⟶^{\overline{b}} ζ_{k} gives a second increasing chain in the réseau.

To see this, first note that β_{k−1} = b is impossible as these are consecutive covers in the Lagrangian order (see relation (iv) of Equation (2)). Define η by ι(η) = ζ and pull this chain back to [e, η]_{≺^{)}} . It is the unique peakless chain in [e, η]_{≺^{)}} and α_{k} is the minimal label. By Lemma 6.8(1), b = min(supp(η)) and so β_{k−1} ≥ b.

Lemma 6.16. Let ζ ∈ B_{∞} and suppose there is an increasing chain in the réseau [e, ζ]_{≺}. Then ζ is minimal. If ζ is a minimal cycle, then there are precisely 2^{δ(ζ)} such chains.

Proof. Let

e ⟶^{β_{1}} ζ_{1} ⟶^{β_{2}} ⋯ ⟶^{β_{m}} ζ_{m} = ζ (11)

be an increasing chain in [e, ζ]_{≺}. Suppose that β_{k−1} < 0 < β_{k}. Then for i < k, δ(ζ_{i}) = 1. Define η_{i} ∈ S_{∞} by ι(η_{i}) = ζ_{i} for i < k. Then

e ⟶^{\overline{β_{1}}} η_{1} ⟶^{\overline{β_{2}}} ⋯ ⟶^{\overline{β_{k−1}}} η_{k−1}

is a decreasing chain in [e, η]_{≺^{)}} , with the lower labeling convention. Let

e ⟶^{α_{1}} ξ_{1} ⟶^{α_{2}} ⋯ ⟶^{α_{k−1}} ξ_{k−1} = η_{k−1}

be the unique decreasing chain in the upper labeling convention. Concatenating the image of this chain in [e, ζ]_{≺} with the end of the chain (11) gives a peakless chain

e ⟶^{α_{1}} ι(ξ_{1}) ⟶^{α_{2}} ⋯ ⟶^{α_{k−1}} ι(ξ_{k−1}) = ζ_{k−1} ⟶^{β_{k}} ⋯ ⟶^{β_{m}} ζ (12)

in the interval [e, ζ]_{≺} in the Lagrangian order. By Lemma 6.12, ζ is necessarily minimal.

Suppose now that ζ is a minimal cycle, then the réseau [e, ζ]_{≺} has an increasing chain and the peakless chain (12) is unique. Consider another increasing chain

e ⟶^{β′_{1}} ζ′_{1} ⟶^{β′_{2}} ⋯ ⟶^{β′_{m}} ζ′_{m} = ζ (13)

and form η′_{i}, ξ′_{i}, α′_{i},and k′ as for the original chain 11). If k = k′, then the chains (11), (13) coincide: The final segments agree, by the uniqueness of (12), as do their initial segments, by Proposition 6.14.

If δ(ζ) = 0, then the minimal label in the peakless chain (12) (either α_{k−1} or β_{k}) corresponds to the cover whose reflection has the form t_{a}. As δ(ζ_{k−1}) = 1, this must be β_{k} and so k = k′ and the chain (11) is the unique increasing chain in the réseau [e, ζ]_{≺}.

Suppose now that δ(ζ) = 1 and k < k′. Since α_{i} = α′_{i} for i < k, β_{i} = β′_{i} for i ≥ k′, and α_{1} > ⋯ > α_{k−1} and β_{k} < ⋯ < β_{m}, we must have k′ = k + 1. But then ξ_{i} = ξ′_{i} for i < k and also ζ_{i} = ζ′_{i} for i ≥ k, and so the two chains (11) and (13) agree except for the label of the cover ζ_{k−1}≺· ζ_{k}. Thus there are at most 2 increasing chains in the réseau [e, ζ]_{≺} and their underlying permutations coincide.

Example 6.17. Suppose η = (1, 2, 5, 3, 4) is a permutation in S_{5}. Consider ζ = ⟨1, 2, 5, 3, 4⟩ = η\overline{η}, a permutation in B_{5}. Figure 8 shows the réseau [e, ζ]_{≺}. In this

Figure 8. The interval [e, ⟨1, 2, 5, 3, 4⟩]_{≺}.

réseau, there are two increasing chains

e ⟶^{\overline{3}} ⟨3, 4⟩ ⟶^{\overline{2}} ⟨2, 3, 4⟩ ⟶^{\overline{1}} ⟨1, 2, 3, 4⟩ ⟶^{5} ⟨1, 2, 5, 3, 4⟩

e ⟶^{\overline{3}} ⟨3, 4⟩ ⟶^{\overline{2}} ⟨2, 3, 4⟩ ⟶^{2} ⟨1, 2, 3, 4⟩ ⟶^{5} ⟨1, 2, 5, 3, 4⟩

which correspond to the unique peakless chain in [e, η]_{≺^{)}} with β_{1} > β_{2} > β_{3} < β_{4}

e ⟶^{4} (3, 4) ⟶^{3} (2, 3, 4) ⟶^{2} (1, 2, 3, 4) ⟶^{5} (1, 2, 5, 3, 4) .

Let I(ζ) count the increasing chains in the réseau [e, ζ]_{≺}.

Lemma 6.18. If η, ζ ∈ B_{∞} are disjoint, then I(η · ζ) = I(η) · I(ζ).

Proof. As with Lemma 6.10, this is a consequence of the analogous bijection concerning increasing words among shuffles of words with disjoint alphabets.

For ζ ∈ B_{∞}, define

χ(ζ) = \{ \begin{matrix}2^{#\{irreducible factors η of ζ with δ(η) = 1\}} & if ζ is minimal \\ 0 & otherwise \end{matrix}

Let D(ζ) enumerate the decreasing chains in the réseau [e, ζ]_{≺}. By Theorem 2.11(6), an increasing chain in ζ becomes a decreasing chain in ζ^{−1}. The following result is now immediate.

Corollary 6.19. For ζ ∈ B_{∞}, χ(ζ) = I(ζ) = D(ζ).

7. The Pieri-type formula

Recall that for ζ ∈ B_{n}, δ(ζ) = 1 if a > 0 implies ζ(a) > 0 and δ(ζ) = 0 otherwise.

Lemma 7.1. Suppose ζ ∈ B_{n} with δ(ζ) = 1. Then there is a quadratic form f on V with f|_{K} ≡ 0 for every K ∈ Y_{ζ}.

Proof. We may suppose that supp(ζ) = [n]; otherwise pull back the quadratic form along the map V ↪ V ⊕ (H_{2})^{⊕t} (t = n − #supp(ζ)) of Section 4. Let E_{}, E^{′}be opposite isotropic flags. Since E_{}, E^{′}are opposite flags, we have V = E′ ; \overline{1} ⊕ E\overline{1}. Let β be the alternating form on V and define a symmetric bilinear form q on V by

q(x, y) := β(x^{+}, y^{−}) + β(y^{+}, x^{−}),

where v^{+}, v^{−} are the projections of v ∈ V to the summands E′ ; \overline{1} and E\overline{1}. This form is non-degenerate. Let f be the associated quadratic form.

Let u ∈ B_{n} with u ≤_{0} ζu and ζu a Grassmannian permutation, as in Remark 2.7. Then there is a k ∈ [n] with i ≤ k ⇒ ζu(i) ≤ \overline{1} and i > k ⇒ ζu(i) ≥ 1. Since

δ(ζ) = 1, u(i) ≤ \overline{1} for i ≤ k and u(i) ≥ 1 for i > k. Let F_{}∈ Y_{u}E_{}⋂ Y_{ω_{0}ζu}E^{′}. Then

dim F\overline{1} ⋂ E\overline{1} ≥ #\{l ≥ 1 | u(l) ≤ \overline{1} \} = k

and

dim F\overline{1} ⋂ E′ ; \overline{1} ≥ #\{l ≥ 1 | ζu(l) ≥ 1\} = n − k.

Thus F\overline{1} = F\overline{1} ⋂ E\overline{1} ⊕ F\overline{1} ⋂ E′ ; \overline{1}. Since F\overline{1} is isotropic for the alternating form β, this decomposition shows it is isotropic for the symmetric form q, proving the lemma.

Corollary 7.2. Suppose ζ = ζ_{1} ⋯ ζ_{s} is the factorization of ζ into irreducible permutations. Suppose exactly r of the ζ_{i} have δ(ζ_{i}) = 1. Then there exist r linearly independent quadratic forms f_{1}, …, f_{r} on V with f_{i}|_{K} ≡ 0 for every K ∈ Y_{ζ}.

Proof. By Theorem 3.4(1), there exist symplectic spaces V_{1}, …, V_{s} with V = V_{1} ⊕ ⋯ ⊕ V_{s} such that K ∈ Y_{ζ} may be written as K = K_{1} ⊕ ⋯ ⊕ K_{s}, where, for i = 1, …, s, K_{i} = K ⋂ V_{i} and K_{i} ∈ Y_{ζ′_{i}}, with ζ′_{i} ∈ B_{n_{i}} shape equivalent to ζ_{i}. (Here, dim V_{i} = 2n_{i}.) We may assume that the ζ_{i} are ordered so that δ(ζ_{i}) = 1 for i = 1, …, r and δ(ζ_{i}) = 0 for i > r. By Lemma 7.1, for each i = 1, …, r, there is a quadratic form f_{i} on V_{i} vanishing on K_{i} ∈ Y_{ζ′_{i}}. The pullback of f_{i} along the orthogonal projection V ↠ V_{i} is a quadratic form on V, also denoted f_{i}. These are the desired forms.

The special Schubert variety Υ_{(m)} of Sp_{2n}ℂ/P_{0} consists of those Lagrangian subspaces which meet a fixed (n+1−m)-dimensional isotropic subspace M of V. In what follows, write Υ_{M} for this Schubert variety, and assume M is in general position in V.

For ζ ∈ B_{n}, define

χ(ζ) = \{ \begin{matrix}2^{#\{irreducible factors η of ζ with δ(η) = 1\}} & if ζ is minimal \\ 0 & otherwise \end{matrix}

For ζ ∈ B_{n}, define W_{ζ} to be the zero locus of the forms f_{1}, …, f_{r} of Corollary 7.2, a complete intersection. Then W_{ζ} is the cone over a subvariety of ℙV of degree 2^{r}. If ζ is minimal, then this degree is χ(ζ).

Corollary 7.3. Let ζ ∈ B_{n} with L(ζ) = m and supp(ζ) = [n]. Let M be a general isotropic (n+1−m)-dimensional subspace of V. Then M ⋂ W_{ζ} = {0} unless ζ is minimal, and in that case, M ⋂ W_{ζ} is χ(ζ) reduced lines.

Proof. Since Sp_{2n}ℂ acts transitively on ℙV, Kleiman’s theorem on the transversality of a general translate [22] will imply the corollary if we show

dim M + dim W_{ζ} ≤ n + 1, (14)

with equality only if ζ is minimal.

By Corollary 7.2, dim W_{ζ} = 2n−r, where r counts the irreducible factors η of ζ with δ(η) = 1. Since n = #supp(ζ), Corollary 6.5 implies that dim M = n + 1 − L(ζ) ≤ 1+∑_{i} δ(ζ_{i}) = 1+r, with equality only if each ζ_{i} is a minimal cycle, establishing (14).

Theorem 7.4. Suppose ζ ∈ B_{n} is minimal and supp(ζ) = [n]. Then a general line ⟨v⟩ in W_{ζ} determines a unique K ∈ Y_{ζ} with v ∈ K.

We deduce the Pieri-type formula from Theorem 7.4. First, define θ(ζ) = 0 if ζ is not minimal, and for ζ minimal, set

θ(ζ) := 2^{#{irreducible factors of ζ}−1}.

Recall that b_{m}^{ζ} was the structure constant corresponding to c_{m}^{ζ} for So_{2n+1}ℂ/B.

Theorem D. (Pieri-type Formula) Let ζ ∈ B_{∞} with L(ζ) = m. Then c_{m}^{ζ} = χ(ζ) and b_{m}^{ζ} = θ(ζ).

By Corollaries 6.13 and 6.19, this implies the chain-theoretic version of the Pieritype formula (Theorem A).

Proof. Let s(ζ) be the number of sign changes in a permutation ζ ∈ B_{∞}. Since s(v_{m}) = 1 and s(ζ) = s(ζu) − s(u) if u ≤_{0} ζu, Equation (4) implies that b_{m}^{ζ} = 2^{s(ζ)−1}c_{m}^{ζ}. Since, for a minimal cycle ζ, s(ζ) + δ(ζ) = 1, we have

θ(ζ) = 2^{s(ζ)−1}χ(ζ).

Thus it suffices to show c_{m}^{ζ} = χ(ζ).

By Theorem B(2), replacing ζ by a shape equivalent permutation if necessary, we may assume that supp(ζ) = [n]. Let M be a general isotropic (n+1−m)-dimensional subspace of V. By the projection formula, c_{m}^{ζ} = deg(Υ_{M} ⋂ Y_{ζ}). Since K ∈ Υ_{M} ⋂ Y_{ζ} implies that K meets M ⋂ W_{ζ} non-trivially, we see that

c_{m}^{ζ} = d · deg(M ⋂ W_{ζ}),

where d counts the K ∈ Y_{ζ} which contain a general line of W_{ζ}, and the degree is taken in ℙV. By Theorem 7.4, d = 1, which completes the proof.

Reduction of Theorem 7.4 to the case of ζ a minimal cycle.

Let ζ = ζ_{1} ⋯ ζ_{s} be the irreducible factorization of ζ. In the notation of the proof of Corollary 7.2, a general 0 ≠ v ∈ W_{ζ} has the form v = v_{1}⊕⋯⊕v_{s}, where 0 ≠ v_{i} ∈ W_{ζ′_{i}} for i = 1, …, s. Moreover v ∈ K ∈ Y_{ζ} if and only if v_{i} ∈ K_{i} ∈ Y_{ζ′_{i}}. Thus is suffices to prove Theorem 7.4 for ζ a minimal cycle. We do this in the following sections.

7.1. Case δ(ζ) = 1.

Theorem 7.5. Let ζ ∈ B_{n} with supp(ζ) = [n] and L(ζ) = n − 1 so that ζ is a minimal cycle with δ(ζ) = 1. Then, for a general 0 ≠ v ∈ W_{ζ}, there is a unique K ∈ Y_{ζ} with v ∈ K.

Proof. Define η ∈ S_{∞} by ι(η) = ζ. Set k := #{a | a < η(a)}. Recall the notation of Section 5.3: Let L, L^{⊥} be complementary Lagrangian subspaces of V, which are identified as linear duals. Define the map Φ_{k} : G_{k}(L) → Sp_{2n}ℂ/B by H ↦ (H +H^{⊥}), where H^{⊥} ⊂ L^{⊥} is the annihilator of H. Define π_{k} : 𝔽ℓ(L) → G_{k}(L) by E_{}↦ E_{k}. By Corollary 5.11, Φ_{k} : X_{η} ⟶^{∼} Y_{ζ} where X_{η} := π_{k}(X_{u}E_{}⋂ X_{(ηu)^{∨}}E^{′}).

Schubert varieties Ω_{ϱ} of the Grassmannian G_{k}(L) are indexed by ordinary partitions (weakly decreasing sequences) ϱ : n − k ≥ ϱ_{1} ≥ ⋯ ≥ ϱ_{k} ≥ 0 [20]. We show

Φ_{k}^{−1} ({K ∈ Sp_{2n}ℂ/B | v ∈ K}) = Ω_{(n−k,1^{k−1})}, (15)

where (n − k, 1^{k−1}) is the hook-shaped partition with first row n − k and first column k. It follows from the projection formula that

deg (Y_{ζ} ⋂{K | v ∈ K}) = deg (X_{η} ⋂ Ω_{(n−k,1^{k−1})})

which is deg(𝔖_{w} · 𝔖_{ω_{0}ηw} · π_{k}^{∗}S_{(n−k,1^{k−1})}), the product in H^{∗}𝔽ℓL. By [31, Theorem 8], this counts the chains in the k-Bruhat order

w ⟶^{β_{1}} w_{1} ⟶^{β_{2}} ⋯ ⟶^{β_{n−1}} ηw

with β_{1} > β_{2} > ⋯ > β_{k} < β_{k+1} < ⋯ < β_{n−1}. The conclusion follows by Lemma 6.8.

To show (15), suppose H ∈ G_{k}(L) and with v ∈ H +H^{⊥}. Let v^{−} be the projection of v into L and v^{+} its projection to L^{⊥}. Then v = v^{−} ⊕ v^{+} and v^{−} ∈ H and v^{+} ∈ H^{⊥} so that H ⊂ (v^{+})^{⊥}. Thus Φ_{k}^{−1}({K | v ∈ K}) = {H | v^{−} ∈ H and H ⊂ (v^{+})^{⊥}}, which is just the Schubert variety Ω_{(n−k,1^{k−1})}.

7.2. Case δ(ζ) = 0.

Theorem 7.6. Let ζ ∈ B_{n} with δ(ζ) = 0 and L(ζ) = n. Then c_{n}^{ζ} = 1.

This completes the proof of Theorem 7.4, as c_{n}^{ζ} = 1 counts the K ∈ Y_{ζ} which meet a generic line in V. We use some geometric constructions to reduce the computation of c_{n}^{ζ} to the cohomology of the classical flag manifold. Let L, L^{⊥} be complementary Lagrangian subspaces of V with L^{⊥} identified with the linear dual of L as in Section 5.3. For 1 ≤ k ≤ n, set

𝔽_{k} := {F_{k−1} ⊂ F_{k} ⊂ L | dim F_{i} = i},

a variety of partial flags in L.

Let ℘ : 𝔽ℓ(L) → 𝔽_{k} be the projection. Then the projections π_{k−1}, π_{k} of 𝔽ℓ(L) to G_{k−1}(L), G_{k}(L) factor through ℘. Let π_{k−1}, π_{k} also denote the projections of 𝔽_{k} to G_{k−1}(L), G_{k}(L). Let Lag(V) denote Sp_{2n}ℂ/P_{0}, the Grassmannian of Lagrangian subspaces of V. Consider the incidence variety Γ:

[figure: 𝔽_{k}; Lag(V); Γ := {(F_{k−1}, F_{k}, K) | F_{k−1} ⊕ F_{k}^{⊥} ⊊ K ⊊ F_{k} ⊕ F_{k−1}^{⊥}}; g; f]

Then Γ is a ℙ^{1}-bundle over 𝔽_{k} and f is genericaly 1-1: The image of f consists of those K with dim K ⋂ L ≥ k − 1 and dim K ⋂ L^{⊥} ≥ n − k, which is an intersection of Schbert varieties. Thus a generic K in this intersection determines g(f^{−1}(K)) = (K ⋂ L, (K ⋂ L^{⊥})^{⊥}) uniquely.

For j ∈ [n] and a (not necessarily strict) partition ϱ with n−j ≥ ϱ_{1} ≥ ⋯ ≥ ϱ_{j} ≥ 0, let σ_{ϱ} ∈ H^{∗}G_{j}(L) be the Schubert class associated to the partition ϱ, as in [20]. We show:

Lemma 7.7. g_{∗}f^{∗}q_{n} = π_{k}^{∗}σ_{(n−k,1^{k−1})} + π_{k−1}^{∗}σ_{(n−k+1,1^{k−2})}.

Let ζ ∈ B_{n} be minimal with δ(ζ) = 0, L(ζ) + n, and supp(ζ) = [n]. We construct a minimal permutation η ∈ S_{n} with ||η|| = n − 1, a k ∈ [n], and w ∈ S_{n} with ηw ◁_{k} w, and ηw ◁̸_{k−1} w. Let X_{η^{−1}} be ℘(X_{ηw} ⋂ X′_{w^{∨}}). Then [X_{η^{−1}}] = ℘_{∗}(𝔖_{η}w · 𝔖_{w^{∨}}). We show

Lemma 7.8. [Y_{ζ}] = f_{∗}g^{∗}[X_{η^{−1}}].

Theorem 7.6 follows from these Lemmas.

c_{n}^{ζ} = deg([Y_{ζ}] · q_{n}) = deg(f_{∗}g^{∗}[X_{η^{−1}}] · q_{n})
= deg([X_{η^{−1}}] · g_{∗}f^{∗}q_{n}),

by the projection formula and Lemma 7.8. By Lemma 7.7, this is

deg (𝔖_{ηw} · 𝔖_{w^{∨}} · (π_{k}^{∗}σ_{(n−k,1^{k−1})} + π_{k−1}^{∗}σ_{(n−k+1,1^{k−2})})) .

Since ηw ◁̸_{k−1} w, only the first term is non-zero. By [31, Theorem 8] and Lemma 6.8, this degree is 1.

Proof of Lemma 7.7 The class q_{n} ∈ H^{∗}Lag(L) is represented by the Schubert variety

Υ_{v} := {K ∈ Lag(V) | v ∈ K} = {K | β(v, K) ≡ 0},

where 0 ≠ v ∈ V and β is the alternating form. Then g_{∗}f^{∗}q_{n} is represented by g(f^{−1}Υ_{v}) which is

{F_{k−1} ⊂ F_{k} | ∃K with v ∈ K and F_{k−1} ⊕ F_{k}^{⊥} ⊊ K ⊊ F_{k} ⊕ F_{k−1}^{⊥}}.

Since V = L ⊕ L^{⊥}, we may write a general v uniquely as v = w ⊕ u with w ∈ L and u ∈ L^{⊥} and so g(f^{−1}Υ_{v}) is a subset of

{F_{k−1} ⊂ F_{k} | w ∈ F_{k}} ⋂ {F_{k−1} ⊂ F_{k} | F_{k−1} ⊂ u^{⊥}}. (16)

This is an intersection of Schubert varieties (in general position if v is general) of codimensions n−k and k −1, respectively. These Schubert varieties have classes π_{k}^{∗}σ_{(n−k)} and π_{k−1}^{∗}σ_{(1^{k−1})}. Since Υ_{v} has codimension n, g(f^{−1}Υ_{v}) equals this intersection if the map g : f^{−1}Υ_{v} → g(f^{−1}Υ_{v}) is finite. Thus

g_{∗}f^{∗}q_{n} = d(π_{k}^{∗}σ_{(n−k)} · π_{k−1}^{∗}σ_{(1^{k−1})}),

where d is the degree of the map g : f^{−1}Υ_{v} → g(f^{−1}Υ_{v}) (which is 0 if the map is not finite).

To compute d, let F_{k−1} ⊂ F_{k} satisfy w ∈ F_{k} and F_{k−1} ⊂ u^{⊥} with F_{k} ⊄ u^{⊥} and w ∉ F_{k−1}. Then F_{k} ⊕ F_{k−1}^{⊥} ⊄ v^{⊥}, and so f(g^{−1}(F_{k−1}, F_{k})) = F_{k} ⊕ F_{k−1}^{⊥} ⋂ v^{⊥}, which shows d = 1. Lastly, the Pieri-type formula [31] in H^{∗}𝔽_{k} shows

π_{k}^{∗}σ_{(n−k)} · π_{k−1}^{∗}σ_{(1^{k−1})} = π_{k}^{∗}σ_{(n−k),1^{k−1})} + π_{k−1}^{∗}σ_{(n−k+1,1^{k−2})}.

Proof of Lemma 7.8 We first make some definitions. Replacing ζ by ζ^{−1} if necessary, we may assume that α > 0 is the unique number with t_{α}ζ≺· ζ, by Lemma 6.6. Since δ(t_{α}ζ) = 1, we define η ∈ S_{n} by ι(η) = t_{α}ζ. Set

k := #{i | i > η(i)} = {i > 0 | ζ(i) > i}.

Let u ∈ B_{n} satisfy u ≤_{0} ζu with ζu a Grassmannian permutation (cf. Remark 2.7). Since δ(ζ) = 0, ζu(n − k + 2) > 0 > ζu(n + 1 − k). Define j ≥ k by ζu(n + 1 − j) = \overline{α}. Then u(n + 1 − j) > 0. Let w ∈ S_{n} be defined by

w(i) = \{ \begin{matrix}u(n + 1 − i) & i < k \\ u(n + 1 − j) & i = k \\ \overline{u(i − k)} & k < i < n + 1 + k − j \\ \overline{u(i + 1 − k)} & n + 1 + k − j ≤ i ≤ n \end{matrix} . (17)

We claim that ηw ◁_{k} w. Since supp(η) = [n], we cannot have both ηw ◁_{k} w and ηw ◁_{k−1} w. Since u <_{0} ι(η)u, condition (1) of Proposition 2.1 is satisfied. For condition (2), as ζu is a Grassmannian permutation,

ηw(1) > ⋯ > ηw(k − 1) and ηw(k + 1) > ⋯ > ηw(n),

so we only need show i < k with ηw(i) < ηw(k) = α implies w(i) < w(k). Let l = n + 1 − i > n + 1 − j. Then ηw(i) = ι(η)u(l) < ι(η)u(n + 1 − j) = α, which implies u(l) < u(n + 1 − j) and hence w(i) < w(k), as u <_{0} ι(η)u.

Example 7.9. Let ζ = ⟨1, 2, 3, 5, 7, 6, \overline{4}] ∈ B_{7}. Then α = 4 and t_{4}ζ = ⟨1, 2, 3, 5, 7, 6, 4⟩≺· ζ so that η = (1, 2, 3, 5, 7, 6, 4) ∈ S_{7}. Here, k = 3. If we set u = \overline{5} \overline{3}6\overline{2} \overline{1}47, then ζu = \overline{7} \overline{5} \overline{4} \overline{3} \overline{2}16 is a Grassmannian permutation and u <_{0} ζu. We see that j = 5 ≥ 3 = k and w = 746 5321 so that ηw = 614 7532 and ηw ◁_{3} w.

Lemma 7.8 is a consequence of the following construction:

Lemma 7.10. Let ζ, u, η, w,and k be as above. Then there exists a commutative diagram

[figure: X_{η^{−1}}; X_{ηw} ⋂ X′_{w^{∨}}; Γ|_{X_{η^{−1}}}; (℘^{∗}Γ)|_{X_{ηw} ⋂ X′_{w^{∨}}}; Y_{ζ}; Y′_{u} ⋂ Y_{ω_{0}ζu}; ℘; ℘; π; g; g; f; h]

where the maps h, f, ℘, and π are isomorphisms on Zariski dense sets.

Since g^{−1}(X_{η^{−1}}) = Γ|_{X_{η^{−1}}} and Y_{ζ} have the same dimension, and the maps are generically 1-1, f(g^{−1}(X_{η^{−1}})) = Y_{ζ}, which proves Lemma 7.8. Proof. We first define an injective map h : (℘^{∗}Γ)|_{X_{ηw}^{∘}} → Y_{ω_{0}ζu}, then show the restriction of h to (℘^{∗}Γ)|_{X_{ηw} ⋂ X′_{w^{∨}}} has image contained in Y′_{u} ⋂ Y_{ω_{0}ζu}. Since the maps π and ℘ are generically 1-1, by the analog of Theorem 3.3 (Theorem 5.1.4 of [3]) for 𝔽ℓ(L), the concluson follows.

Let E_{}∈ 𝔽ℓ(L) be a complete flag and let (F_{}, K) ∈ ℘^{∗}Γ|_{X_{ηw}^{∘}E}, so that F_{}∈ X_{ηw}^{∘}E_{} and K ∈ Lag(V) satisfy

F_{k−1} ⊕ F_{k}^{⊥} ⊊ K ⊊ F_{k} ⊕ F_{k−1}^{⊥}.

Define h(F_{}, K) ∈ Sp_{2n}ℂ/B by

h(F_{}, K)\overline{ı} := \{ \begin{matrix}F_{n+1−i} & n ≥ i ≥ n + 2 − k \\ F_{k−1} + F_{i+k−2}^{⊥} & n + 1 − k ≥ i ≥ n + 2 − j \\ K ⋂(L + E_{n+ζu(i)}) & n + 1 − j ≥ i \end{matrix} .

We show this defines a flag in Y_{ω_{0}ζu}φ_{n}E_{}. Note first that h(F_{}, K)\overline{ı} = φ_{k−1}(F_{})\overline{ı} for i ≥ n + 2 − j. Since ϵ_{k−1}(ηw)(i) = ω_{0}ζu(i) for ≥ n + 2 − j, Lemma 5.9 shows that

dim(φ_{n}E_{})_{a} ⋂ h(F_{}, K)\overline{ı} ≥ #\{l ≥ i | a ≥ ω_{0}ηu(i)\},

for i ≥ n + 2 − j.

We show this for i ≤ n + 1 − j, which shows h(F_{}, K) ∈ Y_{ω_{0}ζu}φ_{n}E_{}. Since ζu is a Grassmannian permutation, and, for a > 0 (φ_{n}E_{})_{a} = L + E_{n−a}^{⊥}, we need only show that

dim K ⋂(L + E_{n+ζu(i)}^{⊥}) ≥ n + 1 − i (18)

for i ≤ n + 1 − j (as ζu(i) < 0 in this range).

It suffice to show this for the dense subset of (F_{}, K) with K ⋂ L = F_{k−1}. Then F_{k−1}^{⊥} is the image of K under the projection V ↠ L. Thus, for a > 0

dim K ⋂(L + E_{n−a}^{⊥}) = k − 1 + dim F_{k−1}^{⊥} ⋂ E_{n−a}^{⊥}
= a + #{k − 1 ≥ l | ηw(l) ≥ a + 1}

If k − 1 ≥ l, then ηw(l) = ζu(n + 1 − l) and these exhaust the positive values of ζu. If i ≤ n + 1 − j and we set a = \overline{ζu(i)}, we see that dim K ⋂(L + E_{n+ζu(i)}^{⊥}) is \overline{ζu(i)} + #\{l | ζu(l) ≥ \overline{ζu(i)} + 1\}. Since ζu ∈ B_{n} is a Grassmannian permutation,

\{\overline{ζu(i)}, …, n\} = \{\overline{ζu(i)}, …, \overline{ζu(1)}\} ∐ \{ζu(l) | ζu(l) > \overline{ζu(i)}\}.

Thus n+1−\overline{ζu(i)} = i+#\{l | ζu(l) ≥ \overline{ζu(i)}+1\} and so dim K ⋂(L+E_{n+ζu(i)}^{⊥}) = n+ 1 − i, which completes the proof that h(F_{}, K) ∈ Y_{ω_{0}ζu}φ_{n}E_{}for (F_{}, K) ∈ g^{−1}X_{ηw}^{∘}E_{}.

We do a useful calculation before we finish the proof of Lemma 7.10.

Lemma 7.11. If F_{}∈ X_{ηw}^{∘}E_{}and 1 ≤ i ≤ n + 1 − j, then F_{k}^{⊥} ⋂ E_{n+ζu(i)}^{⊥} = F_{k+i−1}^{⊥}.

Proof. Similar to the last paragraph, dim F_{k−1}^{⊥} ⋂ E_{n+ζu(i)}^{⊥} = n + 1 − i − k = dim F_{k+i−1}^{⊥}. Thus, we need only show F_{k+i−1}^{⊥} ⊂ F_{k−1}^{⊥} ⋂ E_{n+ζu(i)}^{⊥}.

Note that

dim F_{k+i−1} ⋂ E_{n+ζu(i)} = #\{k + i − 1 ≥ l | ηw(l) ≥ \overline{ζu(i)} + 1\}

Since ηw(k + 1) > ⋯ > ηw(n) and ηw(k + i) = \overline{ζu(i)}, this equals

#{k + i > l | ηw(l) > ηw(k + i)} = n − ηw(k + i) = n + ζu(i),

which shows E_{n+ζu(i)} ⊂ F_{k+i−1}. Since F_{k} ⊂ F_{k+i−1}, this completes the proof.

We complete the proof of Lemma 7.10, showing that if we further require F_{}∈ X′_{w^{∨}}, then h(F_{}, K) ∈ Y′_{u}. Let E^{′}be a flag opposite to E_{}in L. We show h(F_{}, K) ∈ Y_{u}φ_{0}E^{′} for (F_{}, K) ∈ (℘^{∗}Γ)|_{X_{ηw} ⋂ X′_{w^{∨}}}. Note first that F_{}satisfies

dim E′_{a} ⋂ F_{b} = #{b ≥ l | w(l) ≤ a}.

As before

dim(φ_{0}E^{′})_{a} ⋂ h(F_{}, K)\overline{ı} ≥ #\{i ≤ l | a ≥ u(l)\} (19)

for i > n + 1 − j since ϵ_{k−1}(w^{∨}) and u agree in this range, as do h(F_{}, K)\overline{ı} and (φ_{k−1}F_{})\overline{ı}. For i ≤ n + 1 − j, it suffices to establish (19) for the dense subset of those (F_{}, K) with K ⋂ L^{⊥} = F_{k}^{⊥}.

Suppose a > 0. Since K ⋂ L^{⊥} = F_{k}^{⊥}, F_{k} is the image of K under the projection V ↠ L. Similarly, F_{k} is the image of K ⋂(L + E_{n+\overline{ζu(i)}}^{⊥}) = h(F_{}, K)\overline{ı} under this projection. Since the kernel of this is F_{k}^{⊥} ⋂ E_{n+ζu(i)}^{⊥} = F_{k+i−1}^{⊥} and (φ_{0}E^{′})_{a} = (L^{⊥} + E′_{a}),

dim(φ_{0}E^{′})_{a} ⋂ h(F_{}, K)\overline{ı} = dim F_{k+i−1}^{⊥} + dim F_{k} ⋂ E′_{a},
= n + 1 − k − i + #{k ≥ l | w(l) ≤ a},
= n + 1 − i − #{k ≥ l | w(l) > a}.

Since {w(1), …, w(k)} = {u(n), …, u(n+2−k), u(n+1−j)} are the positive values of u and i ≤ n + 1 − j, this is

n + 1 − i − #{i ≤ l | u(l) > a} = #{i ≤ l | u(l) ≤ a},

which shows (19).

Now suppose a < 0. Then (φE^{′})_{a} = (E′_{\overline{a}−1})^{⊥} ⊂ L^{⊥} and so

(φ_{0}E^{′})_{a} ⋂ h(F_{}, K)\overline{ı} = F_{k}^{⊥} ⋂ E_{n+\overline{ζu(i)}}^{⊥} ⋂(E′_{\overline{a}−1})^{⊥}
= F_{k+i−1}^{⊥} ⋂(E′_{\overline{a}−1})^{⊥}.

This has dimension

n − k − i + 2 − \overline{a} + #\{k + i − 1 ≥ l | w(l) ≤ \overline{a} − 1\} = #\{k + i ≤ l | w(l) ≥ \overline{a}\}.

The values \overline{w(l)} for l ≥ k + i are simply the negative values u(l) for l ≥ i. Thus this is #{i ≤ l | u(l) ≤ a}, which completes the proofs of Lemma 7.10 and the Pieri-type formula.

References

[1] N. Bergeron and S. Billey, RC-Graphs and Schubert polynomials, Experimental Math., 2 (1993), pp. 257–269.

[2] N. Bergeron, S. Mykytiuk, F. Sottile, and S. van Willigenburg, P -partitions for edge-labeled posets. in preparation, 1998.

[3] N. Bergeron and F. Sottile, Schubert polynomials, the Bruhat order, and the geometry of flag manifolds. Duke Math. J., to appear, 1998.

[4] , A monoid for the Grassmann-Bruhat order. MSRI preprint 1997-073, math.CO/9712258, 1997.

[5] , Hopf algebras of edge-labeled posets. math.CO/9712256, 1997.

[6] I. N. Bernstein, I. M. Gelfand, and S. I. Gelfand, Schubert cells and cohomology of the spaces G/P, Russian Mathematical Surveys, 28 (1973), pp. 1–26.

[7] S. Billey and M. Haiman, Schubert polynomials for the classical groups, J. AMS, 8 (1995), pp. 443–482.

[8] B. Boe and H. Hiller, Pieri formula for SO_{2n+1}/U_{n} and SP_{n}/U_{n}, Adv. in Math., 62 (1986), pp. 49–67.

[9] A. Borel, Sur la cohomologie des espaces fibrés principaux et des espaces homogènes des groupes de Lie compacts, Ann. Math., 57 (1953), pp. 115–207.

[10] C. Chevalley, Sur les décompositions cellulaires des espaces G/B, in Algebraic Groups and their Generalizations: Classical Methods, American Mathematical Society, 1994, pp. 1–23. Proceedings and Symposia in Pure Mathematics, vol. 56, Part 1.

[11] P. Deligne, Cohomologie Etale, SGA4\frac{1}{2}, no. 569 in Springer Lecture Notes, Springer-Verlag, 1977.

[12] M. Demazure, Désingularization des variétés de Schubert généralisées, Ann. Sc. E. N. S. (4), 7 (1974), pp. 53–88.

[13] V. Deodhar, On some geometric aspects of Bruhat orderings. I. A finer decomposition of Bruhat cells, Invent. Math., 79 (1985), pp. 499–511.

[14] D. Edidin and W. Graham, Characteristic classes and quadric bundles, Duke Math. J., 78 (1995), pp. 277–299.

[15] S. Fomin, Duality of graded graphs, J. Alg. Combin., 3 (1994), pp. 357–404.

[16] , Schensted algorithms for dual graded graphs, J. Alg. Combin., 4 (1994), pp. 1–45.

[17] S. Fomin and A. N. Kirillov, Combinatorial B_{n}-analogs of Schubert polynomials, Trans. AMS, 348 (1996), pp. 3591–3620.

[18] W. Fulton, Intersection Theory, no. 2 in Ergebnisse der Math., Springer-Verlag, 1984.

[19] , Determinantal formulas for orthogonal and symplectic degeneracy loci, J. Diff. Geom., 43 (1996), pp. 276–290.

[20] , Young Tableaux, Cambridge University Press, 1996.

[21] T. Józefiak, Schur Q-functions and the cohomology of isotropic Grassmannians, Math. Proc. Camb. Phil. Soc., 109 (1991), pp. 471–8.

[22] S. Kleiman, The transversality of a general translate, Comp. Math., 28 (1974), pp. 287–297.

[23] G. Kreweras, Sur les partitions non croissees d’un cycle, Discrete Math., 1 (1972), pp. 333– 350.

[24] V. Lakshmibai and C. S. Seshadri, Geometry of G/P II, Proc. Ind. Acad. Sci., 87 (1978), pp. 1–54.

[25] A. Lascoux and M.-P. Schützenberger, Polynômes de Schubert, C. R. Acad. Sci. Paris, 294 (1982), pp. 447–450.

[26] , Symmetry and flag manifolds, in Invariant Theory, (Montecatini, 1982), vol. 996 of Lecture Notes in Math., Springer-Verlag, 1983, pp. 118–144.

[27] I. G. Macdonald, Symmetric Functions and Hall Polynomials, Oxford University Press, 1995. second edition.

[28] P. Pragacz, Algebro-geometric applications of Schur S-and Q-polynomials, in Topics in Invariant Theory, Séminaire d’Algèbre Dubreil-Malliavin 1989-90, Springer-Verlag, 1991, pp. 130–191.

[29] P. Pragacz and J. Ratajski, Formulas for Lagrangian and orthogonal loci: The Q̃- polynomial approach. Composito Math., to appear, 1997.

[30] V. Reiner, Non-crossing partitions for the classical reflection groups, Discrete Math., 177 (1997), pp. 195–222.

[31] F. Sottile, Pieri’s formula for flag manifolds and Schubert polynomials, Annales de l’Institut Fourier, 46 (1996), pp. 89–110.

[32] , Pieri-type formulas for maximal isotropic Grassmannians via triple intersections. MSRI preprint 1997-062, alg-geom 9708026, 1997.

[33] J. Stembridge, Shifted tableaux and the projective representations of the symmetric group, Adv. Math., 74 (1989), pp. 87–134.

[34] , Enriched P -partitions, Trans. AMS, 349 (1997), pp. 763–788.

Department of Mathematics and Statistics, York University, Toronto, Ontario M3J 1P3, CANADA

E-mail address, Nantel Bergeron: bergeron@mathstat.yorku.ca

  URL, Nantel Bergeron: http://www.math.yorku.ca/bergeron

Department of Mathematics, University of Wisconsin, Van Vleck Hall, 480 Lincoln Drive, Madison, Wisconsin 53706-1388, USA

E-mail address, Frank Sottile: sottile@math.wisc.edu

  URL, Frank Sottile: http://www.math.wisc.edu/~sottile