MSRI Series Volume 28, 1995

Vector Bundles and Brill–Noether Theory

SHIGERU MUKAI

Abstract. After a quick review of the Picard variety and Brill–Noether theory, we generalize them to holomorphic rank-two vector bundles of canonical determinant over a compact Riemann surface. We propose several problems of Brill–Noether type for such bundles and announce some of our results concerning the Brill–Noether loci and Fano threefolds. For example, the locus of rank-two bundles of canonical determinant with five linearly independent global sections on a non-tetragonal curve of genus 7 is a smooth Fano threefold of genus 7.

As a natural generalization of line bundles, vector bundles have two important roles in algebraic geometry. One is the moduli space. The moduli of vector bundles gives connections among different types of varieties, and sometimes yields new varieties that are difficult to describe by other means. The other is the linear system. In the same way as the classical construction of a map to a projective space, a vector bundle gives rise to a rational map to a Grassmannian if it is generically generated by its global sections. In this article, we shall describe some results for which vector bundles play such roles. They are obtained from an attempt to generalize Brill–Noether theory of special divisors, reviewed in Section 2, to vector bundles. Our main subject is rank-two vector bundles with canonical determinant on a curve C with as many global sections as possible: especially their moduli and the Grassmannian embeddings of C by them (Section 4).

1. Line bundles

Let X be a smooth algebraic variety over the complex number field C. We consider the set of isomorphism classes of line bundles, or invertible sheaves, on X. This set enjoys two good properties, neither of which holds anymore for vector bundles of higher rank. One is that it has a natural algebraic structure as a moduli space without any modification. The other is that it becomes a (commutative) group by the tensor product. In fact, the isomorphism classes are parametrized by the first cohomology group H^{1}(O_{X}^{∗}) with coefficient in the (multiplicative) sheaf of nowhere vanishing holomorphic functions. H^{1}(O_{X}^{∗}) endowed with the natural algebraic structure is called the Picard variety and denoted by Pic X. Let

Based on the author’s three talks given at JAMI in 1991, UCLA in 1992 and Durham University in 1993. Supported in part by a Grant under The Monbusho International Science Research Program: 04044081.

⋯ ⟶ H^{1}(X, Z) ⟶ H^{1}(O_{X}) ⟶ H^{1}(O_{X}^{∗}) ⟶^{δ} H^{2}(X, Z) ⟶ ⋯ (1.1)

be the long exact sequence derived from the exponential exact sequence

0 ⟶ Z ⟶^{2πi} O_{X} ⟶^{exp} O_{X}^{∗} ⟶ 1 (1.2)

of sheaves on X. The connecting homomorphism δ associates the first Chern class c_{1}(L) to each line bundle [L] ∈ H^{1}(O_{X}^{∗}). For example, if X is a curve, δ(L) is the degree of L under the natural identification H^{2}(X, Z) ≃ Z. By (1.1), the neutral component Pic^{0} X of Pic X is isomorphic to the quotient group H^{1}(O_{X})/H^{1}(X, Z), which is an abelian variety if X is a projective variety.

Let C be a curve, or a compact Riemann surface, of genus g. The Riemann– Roch theorem

\{ \begin{matrix}χ(L) := h^{0}(L) − h^{1}(L) = deg L + 1 − g, \\ H^{1}(L) ≃ H^{0}(K_{C}L^{−1})^{∨}, \end{matrix} (1.3)

is most fundamental for its study. The latter isomorphism is functorial in L and referred to as the Serre duality. By (1.1), Pic C is the disjoint union of Pic^{d} C, for d ∈ Z, where Pic^{d} C is the set of isomorphism classes of line bundles of degree d. By (1.3), the number h^{0}(L) of linearly independent global sections is constant on Pic^{d} C unless 0 ≤ d ≤ 2g −2 = deg K_{C}. Conversely, when 0 ≤ d ≤ 2g −2, the number h^{0}(L) is equal to max{0, d + 1 − g} on a non-empty Zariski open subset of Pic^{d} C, but not constant since there exists a special line bundle, that is, a line bundle L with h^{0}(L)h^{1}(L) ≠ 0, of degree d. The space Pic^{d} C is stratified by h^{0}(L). For an integer r ≥ max{0, d + 1 − g}, we set

W_{d}^{r}(C) = {[L] | h^{0}(L) ≥ r + 1} ⊂ Pic^{d} C,

which is closed in the Zariski topology. The case (d, r) = (g − 1, 0) is most important. W_{g−1}^{0} is a divisor and usually denoted by Θ. The self-intersection number (Θ^{g}) is equal to g!, i.e., Θ is a principal polarization of Pic^{g−1} C. This principally polarized abelian variety (Pic^{g−1} C, Θ) is called the Jacobian of C.

Often the isomorphism class of C is recovered from the variety W_{d}^{r}(C) of special line bundles. The case of theta divisor Θ is classical:

Theorem 1.4 (Torelli). Two curves are isomorphic to each other if their Jacobians are so.

We refer to [13] for various approaches to this important result. Let C be a non-hyperelliptic curve of genus 5. Then W_{4}^{1}(C) is a curve of genus 11. (If C is trigonal or W_{4}^{1}(C) contains a line bundle with L^{2} ≃ K_{C}, then W_{4}^{1}(C) is singular. But still the theorem holds true.)

Another example is:

Theorem 1.5. The Jacobian of C is isomorphic to the Prym variety of

(W_{4}^{1}(C), σ),

where σ is the involution of Pic^{4} C defined by σ[L] = [K_{C}L^{−1}].

See [2] for the proof in the case where C is a complete intersection of three quadrics in P^{5}.

Another feature of special line bundles is their relation with projective embeddings. If a line bundle L is generated by its global sections, we obtain a morphism

Φ_{|L|} : C ⟶ P^{∗}H^{0}(L),

where P^{∗}H^{0}(L) is the projectivization of the dual vector space of H^{0}(L). The most interesting case is K_{C}, the canonical line bundle, which appears in (1.3). By the Riemann–Roch theorem, K_{C} is generated by global sections, and Φ_{|K|} : C ⟶ P^{∗}H^{0}(K_{C}) = P^{g−1} is an embedding unless C is hyperelliptic. The image C_{2g−2} ⊂ P^{g−1} of Φ_{|K|} is called the canonical model of C. Here is a classical example:

Theorem 1.6 (Enriques–Petri). The canonical model C_{2g−2} ⊂ P^{g−1} is an intersection of quadrics if and only if W_{3}^{1}(C) = W_{5}^{2}(C) = ∅.

We refer to [1] and [6] for further results of this kind. The latter also discusses an interesting use of vector bundles that we do not treat here.

2. Brill–Noether theory

We study W_{d}^{r}(C) more closely. First we note that it is not only a subset but a subscheme of Pic C. Take distinct points P_{1}, …, P_{N} of C and put D = ∑_{i=1}^{N} P_{i}. Choose N sufficiently large so that H^{1}(L(D)) vanishes for every [L] ∈ Pic^{d} C. The exact sequence

0 ⟶ L ⟶ L(D) ⟶^{res} ⨁_{i=1}^{N} L(D)|_{P_{i}} ⟶ 0 (2.1)

of sheaves on C induces the exact sequence

0 ⟶ H^{0}(L) ⟶ H^{0}(L(D)) ⟶^{H^{0}(res)} ⨁_{i=1}^{N} H^{0}(L(D)|_{P_{i}}) ⟶ H^{1}(L) ⟶ 0 (2.2)

of vector spaces. There exists a homomorphism R : E ⟶ F between two vector bundles E and F on Pic^{d} C whose fibre R_{[L]} at [L] is the above H^{0}(res) for every [L] ∈ Pic^{d} C (these bundles are the direct images of certain sheaves on C × Pic C). The difference in rank between E and F does not depend on D: we have

r(F) − r(E) = N − h^{0}(L(D)) = g − 1 − d

by (1.3). The following statement is easy to verify:

Lemma 2.3. Let E and F be finite-dimensional vector spaces, let c be a positive integer, and set W = {f ∈ Hom(E, F) | dim Ker f ≥ c}. Then:

1) W is a closed subvariety of codimension max{0, c(c + δ)} in the affine space Hom(E, F), where δ = dim F − dim E, and

2) if dim Ker f = c, then W is smooth at the point f and the normal space N_{W/ Hom,f} is isomorphic to Hom(Ker f, Coker f).

Since W_{d}^{r}(C) is

{α ∈ Pic^{d} C | dim Ker R_{α} ≥ r + 1},

it is a closed subscheme of Pic^{d} C and its codimension is at most (r+1)(g+r−d) by the lemma. It follows that

dim W_{d}^{r}(C) ≥ g − (r + 1)(g + r − d). (2.4)

For a line bundle L on C, we put ρ(L) = g − h^{0}(L)h^{1}(L) and call it the Brill– Noether number. When [L] ∈ W_{d}^{r}(C) and h^{0}(L) = r + 1, this number is equal to the right-hand side of the above inequality. Since the tangent space of Pic C is isomorphic to H^{1}(O_{C}) by (1.1), the Zariski tangent space of W_{d}^{r}(C) at [L] is the kernel of the tangential map H^{1}(O_{C}) ⟶ Hom(H^{0}(L), H^{1}(L)) by (2.2) and Lemma 2.3(2).

Now we describe the Zariski tangent space more directly. Let

τ_{L} : H^{1}(O_{C}) ⟶ Hom(H^{0}(L), H^{1}(L))

be the linear map induced by the cup product H^{1}(O_{C}) × H^{0}(L) ⟶ H^{1}(L). By the Serre duality (1.3), the dual of τ_{L} is the multiplication map

H^{0}(L) ⊗ H^{0}(K_{C}L^{−1}) ⟶ H^{0}(K_{C}), (2.5)

called the Petri map.

Proposition 2.6. Assume that [L] ∈ W_{d}^{r}(C) and h^{0}(L) = r + 1. Then:

1) the Zariski tangent space of W_{d}^{r}(C) at [L] is isomorphic to Ker τ_{L},

2) dim Ker τ_{L} ≥ dim_{[L]} W_{d}^{r}(C) ≥ ρ(L), and

3) the following three conditions are equivalent:

i) W_{d}^{r}(C) is smooth and of dimension ρ(L) at [L], ii) τ_{L} is surjective, and iii) the Petri map (2.5) is injective,

Proof. Let α = {a_{ij}} ∈ H^{1}(O_{C}^{∗}) be the cohomology class corresponding to L, that is, a_{ij} ∈ O_{U_{i}∩U_{j}} are the transition functions of L for a suitable open covering {U_{i}} of C. Let ε be the dual number, i.e., ε ≠ 0 but ε^{2} = 0. A first-order infinitesimal deformation L̃ of L corresponds to a cohomology class α̃ = {ã_{ij}} ∈ H^{1}(O_{C}[ε]^{∗}) whose reduction modulo ε is α. Also, α̃ is of the form {a_{ij}(1 + b_{ij}ε)} for β = {b_{ij}} ∈ H^{1}(O_{C}). Let h ∈ H^{0}(L) be a global section of L; then h is a collection {h_{i}} of h_{i} ∈ O_{U_{i}} such that h_{i} = a_{ij}h_{j}. The differences b_{ij}h_{j}ε of h_{i} and ã_{ij}h_{j} form a one-cocycle whose cohomology class is the cup product (β^{∪}h)ε. Hence h extends to a global section of L̃ if and only if β^{∪}h = 0 in H^{1}(L). Therefore, all global sections of L extend if and only if the cup product map ^{∪}β : H^{0}(L) ⟶ H^{1}(L) is zero, which shows (1). Part (2) follows from (1) and (2.4). Part (3) is straightforward from (2). □

Let ρ be the right-hand side of (2.4). We refer to [1] for the following important results:

Theorem 2.7 (Kempf Kleiman–Laksov; Fulton–Lazarsfeld).

(Existence) W_{d}^{r}(C) ≠ ∅ if ρ ≥ 0.

(Connectedness) W_{d}^{r}(C) is connected if ρ > 0.

Let M_{g} be the moduli space of curves of genus g.

Theorem 2.8 (Gieseker [5], Lazarsfeld [7]). If [C] ∈ M_{g} is general, the Petri map (2.5) is injective for every (special) line bundle L on C.

In particular, W_{d}^{r}(C) is of dimension ρ if ρ is nonnegative, and empty otherwise. Thus the estimate (2.4) is best possible for the generic curve. When ρ = 0, the number of W_{d}^{r}(C) is finite and was first computed by Castelnuovo. Let G(a, a + b) ⊂ P_{∗} ⋀^{a} C^{a+b} be the Plücker embedding of the Grassmannian of a-dimensional subspaces of C^{a+b}. The following result is interesting (cf. (4.15)):

Theorem 2.9. If [C] ∈ M_{g} is general and ρ = 0, the number of W_{d}^{r}(C) is equal to the degree of the g-dimensional Grassmannian

G(a, a + b) ⊂ P_{∗} ⋀^{a} C^{a+b},

where a = r + 1 and b = g + r − d.

In fact, both #W_{d}^{r}(C) and deg G(a, a + b) are equal to

g! ∏_{1≤i≤a<j≤a+b}(j − i)^{−1}.

3. Vector bundles on a curve

Let X be a smooth complete algebraic variety (over C). By O_{X}, we mean either the sheaf of holomorphic functions in the usual topology or the sheaf of regular functions in the Zariski topology. In the study of vector bundles, this is allowed by virtue of the GAGA principle, which says that the two categories of analytic and algebraic coherent sheaves on X are equivalent to each other. By a vector bundle E on X we mean a locally free O_{X}-module. There exists an open covering {U_{i}} of X such that E |_{U_{i}}≃ O_{U_{i}}^{⊕r} for every i. This positive integer r is called the rank of E. The highest exterior product ⋀^{r} E is a line bundle on X, denoted by det E.

Assume that E is generated by global sections, that is, the evaluation homomorphism

ev_{E} : H^{0}(E) ⊗ O_{X} ⟶ E

is surjective. Then every fibre E_{x} of E is an r-dimensional quotient space of H^{0}(E). Hence we obtain a map Φ_{|E|} : X ⟶ G(H^{0}(E), r) to the Grassmannian of r-dimensional quotient spaces of H^{0}(E). This map is holomorphic since E is so. The exterior product

⋀^{r} ev_{E} : ⋀^{r} H^{0}(E) ⊗ O_{X} ⟶ ⋀^{r} E

of ev_{E} induces a linear map

⋀^{r} H^{0}(E) ⟶ H^{0}(⋀^{r} E), (3.1)

which we denote by λ_{E}. The exterior product ⋀^{r} E_{x} of fibres E_{x} are quotient spaces of both H^{0}(⋀^{r} E) and ⋀^{r} H^{0}(E). The former determines a point in the projective space P^{∗}H^{0}(⋀^{r} E) and the latter the Plücker coordinate of [E_{x}] ∈ G(H^{0}(E), r). Hence we obtain the following commutative diagram:

X  ⟶^{Φ_{|E|}} G(H^{0}(E), r)
Φ_{|∧^{r}E|} ↓  ∩ Plücker embedding
P^{∗}H^{0}(⋀^{r} E) ⋯^{P^{∗}λ_{E}}→ P^{∗} ⋀^{r} H^{0}(E).

(3.2)

Thus λ_{E} connects the projective and Grassmannian embeddings. This map is important in other considerations, too. See (3.5), (4.7) and (4.15).

Let A_{ij} ∈ GL(r, O_{U_{i}∩U_{j}}) be the transition matrix function of a vector bundle E, that is, the matrix expression of the composite of the two isomorphisms O_{U_{j}}^{⊕r} ≃ E|_{U_{j}} and E|_{U_{i}} ≃ O_{U_{i}}^{⊕r} over U_{i} ∩ U_{j}. The collection {A_{ij}} of all such (matrix) functions is a one-cocycle with coefficients in the sheaf GL(r, O_{X}) of noncommutative groups. Hence the rank-r vector bundles on X are parametrized by the first cohomology set H^{1}(GL(r, O_{X})). The determinant homomorphism det : GL(r, O_{X}) ⟶ O_{X}^{∗} induces the map

H^{1}(det) : H^{1}(GL(r, O_{X})) ⟶ H^{1}(O_{X}^{∗}) = Pic X,

whose fibre at [L] ∈ Pic X is denoted by B_{X}(r, L). Moduli Problem. a) Give a natural algebraic structure to a suitable open

subset of B_{X}(r, L) and construct its geometric compactification.

b) What properties of (X, L) are inherited by the moduli space constructed in (a)? The fibre B_{X}(r, L) does not have a nice description such as Pic X in (1.1). But

its Zariski tangent space as a functor is easy to identify. Let E be a vector bundle and {A_{ij}} the one-cocycle of transition functions. A one-cochain {A_{ij}(I_{r}+B_{ij}ε)} with values in GL(r, O_{X}[ε]) is a cocycle if and only if

A_{jk}^{−1}B_{ij}A_{jk} + B_{jk} = B_{ik}

holds on U_{i} ∩ U_{j} ∩ U_{k} for every i, j and k, where ε is the dual number. This is the same as saying that {B_{ij}} is a one-cocycle with values in the sheaf End E ≃ E^{∨} ⊗ E of (local) endomorphisms of E. By this correspondence, the first-order infinitesimal deformations of E are parametrized by the cohomology group H^{1}(End E). Since

det(A_{ij}(I_{r} + B_{ij}ε)) = (det A_{ij})(I_{r} + tr B_{ij}ε),

the Zariski tangent space of B_{X}(r, L) is isomorphic to the kernel of

H^{1}(tr) : H^{1}(End E) ⟶ H^{1}(O_{X}),

which is the tangential map of H^{1}(det). Let sl(E) be the sheaf of traceless endomorphisms of E. Then End E is the direct sum of two vector bundles sl(E) and O_{X}. Therefore, the Zariski tangent space is isomorphic to H^{1}(sl(E)).

Let C be a curve of genus g. The Riemann–Roch theorem (1.3) is generalized to the formula

χ(E) := h^{0}(E) − h^{1}(E) = deg E + r(1 − g)
H^{1}(E) ≃ H^{0}(K_{C}E^{∨})^{∨} (Serre duality),

(3.3) where deg E is the degree of det E. For simplicity, we restrict ourselves to the case r = 2. The answer to part (a) of the moduli problem is the notion of stability: Definition 3.4. (Mumford [11]) A rank-two vector bundle E on C is stable if deg ξ < \frac{1}{2} deg E for every line subbundle ξ of E. It is semi-stable if deg ξ ≤ \frac{1}{2} deg E for every ξ.

Let L be a line bundle on C and E a member of B_{C}(2, L). Fix an ample line bundle O_{C}(1) on C. Then E(n) belongs to B_{C}(2, L(2n)) and we obtain the linear map

λ_{E(n)} : ⋀^{2} H^{0}(E(n)) ⟶ H^{0}(L(2n)) (3.5)

as in (3.1). The above condition deg ξ < \frac{1}{2} deg E is equivalent to the asymptotic stability of the linear map λ_{E(n)} with respect to the action of the special linear group SL(H^{0}(E(n))) [4]. By the geometric invariant theory, the (coarse) moduli space M_{C}(2, L) of stable two-bundles with determinant L exists as a quasiprojective algebraic variety. Moreover, it becomes a projective algebraic variety \overline{M}_{C}(2, L) by adding certain equivalence classes of semi-stable two-bundles.

Every line bundle M induces an isomorphism M_{C}(2, L) ≃ M_{C}(2, LM^{2}) taking E to E⊗M. Hence there are only two isomorphism classes of the moduli space of two-bundles: M_{C}(2, odd) and M_{C}(2, even). Both are smooth and of dimension 3g − 3 = dim H^{1}(sl(E)). Since every semi-stable two-bundle of odd degree is stable, M_{C}(2, odd) is a projective variety. Among many known global properties of M_{C}(2, odd), we state two. One is

Theorem 3.6 (Ramanan [19]). M_{C}(2, odd) is a Fano manifold of index two.

A smooth projective variety X is called a Fano manifold if the anti-canonical line bundle K_{X}^{−1} = det T_{X} is ample. The largest integer that divides c_{1}(X) in H^{2}(X, Z) is called the index. In the case of M_{C}(2, odd), the Picard group is free cyclic and the anti-canonical line bundle is the square of the positive generator. Example. (Desale and Ramanan [21], Newstead [17]) Let C be a curve of genus two defined by the equation y^{2} = (x − λ_{1})(x − λ_{2}) ⋯ (x − λ_{6}). Then the moduli space M_{C}(2, odd) is the complete intersection

∑_{i=1}^{6} x_{i}^{2} = ∑_{i=1}^{6} λ_{i}x_{i}^{2} = 0

in P^{5}. The anti-canonical line bundle is the square of the restriction of tautological line bundle.

The other result we cite is a Torelli-type theorem:

Theorem 3.7 (Mumford and Newstead [15]). The intermediate Jacobian

H^{2}(Ω_{M}^{1})/H^{3}(M, Z)

of the moduli space M_{C}(2, odd) is isomorphic to the Jacobian of C as polarized abelian variety.

4. Toward a Brill–Noether type theory for two-bundles

M_{C}(2, O_{C}) and M_{C}(2, K_{C}) are two natural representatives of M_{C}(2, even). The following is fundamental for the former:

Theorem 4.1 (Narasimhan and Seshadri [16], Donaldson [3]). There is a natural bijection between the two sets

1) M_{C}(2, O_{C}), the set of isomorphism classes of stable two-bundles with trivial determinant, and

2) Hom^{irr}(π_{1}(C), SU(2))/ SU(2), the set of conjugacy classes of irreducible twodimensional special unitary representations of the fundamental group of C.

We take M_{C}(2, K) to develop a Brill–Noether type theory. See [20] and [18] for another direction of development. The moduli space M_{C}(2, K) of stable twobundles with canonical determinant is stratified by the number h^{0}(E) of linearly independent global sections. By analogy with W_{d}^{r}, we set

M_{C}(2, K, n) = {[E] | h^{0}(E) ≥ n + 2} ⊂ M_{C}(2, K).

We denote by \overline{M}_{C}(2, K, n) the union of M_{C}(2, K, n) and the set of isomorphism classes of semi-stable vector bundles [ξ ⊕ K_{C}ξ^{−1}] ∈ \overline{M}_{C}(2, K) with [ξ] ∈ W_{g−1}^{n/2}(C).

By the same argument as in Section 2, M_{C}(2, K, n) is a closed subscheme of M_{C}(2, K). (The universal family does not exist on C × M_{C}(2, K), but this does not cause a problem for the study of such local properties of the moduli.) A similar consideration gives the estimate dim M_{C}(2, K, n) ≥ 3g − 3 − (n + 2)^{2}. But this estimate is not sharp. The proper one is

Theorem 4.2. dim M_{C}(2, K, n) ≥ 3g − 3 − \frac{1}{2}(n + 2)(n + 3).

Before giving the proof, we recall some notions of symplectic geometry. Let W be a 2ν-dimensional vector space with a non-degenerate skew-symmetric bilinear form ⟨ , ⟩ : W × W ⟶ C. A ν-dimensional subspace V of W is a Lagrangian of W if the bilinear form ⟨ , ⟩ is identically zero on V × V. We denote the set of Lagrangians by L(W), which is a subset of the ν^{2}-dimensional Grassmannian G(ν, W). Fix a Lagrangian U_{∞} and set Z = {[U] | U ∩ U_{∞} = 0} in G(ν, W). When [U_{0}] ∈ Z is fixed, Z is a ν^{2}-dimensional affine space by the bijection

Hom(U_{0}, U_{∞}) ∋ f ↦ Γ_{f} ∈ Z, (4.3) where Γ_{f} ⊂ U_{0} × U_{∞} is the graph of f. Assume that U_{0} is also a Lagrangian. Then U_{0} and U_{∞} are dual under the pairing ⟨ , ⟩. Γ_{f} is a Lagrangian of W if and only if f ∈ Hom(U_{0}, U_{∞}) ≃ U_{∞} ⊗ U_{∞} is symmetric. Hence L(W) is a smooth subvariety of dimension \frac{1}{2}ν(ν + 1) in the Grassmannian G(ν, W).

Proposition 4.4. Fix a Lagrangian [V_{0}] ∈ L(W) and let c be a positive integer. Then the Schubert subvariety

L(W)_{c} = {[V] ∈ L(W) | dim V ∩ V_{0} ≥ c}

is of codimension \frac{1}{2}c(c + 1) in L(W).

Proof. For [V] ∈ L(W), choose a Lagrangian V_{∞} so that V ∩V_{∞} = V_{0}∩V_{∞} = 0. Then V corresponds to a symmetric matrix of size ν via (4.3) and dim V ∩ V_{0} is equal to the co-rank of the symmetric matrix. Hence we have our assertion.□

Proof of Theorem 4.2. Let E be a two-bundle with canonical determinant and D an effective divisor on C. Since E is self-Serre adjoint, that is, E^{∨}K_{C} ≃ E, the two vector spaces H^{0}(E) and H^{1}(E) are dual by (3.3). Similarly, H^{0}(E(−D)) and H^{1}(E(D)) are dual. Hence, by the Riemann–Roch theorem (3.3), we have

h^{0}(E(D)) − h^{0}(E(−D)) = χ(E(D)) = 2N, (4.5)

where N = deg D. Now we denote the quotient sheaf E(D)/E(−D) by A, which is supported by a finite set and has length 4N. We consider the composite of the pairing A × A ⟶ K_{C}(2D)/K_{C}, induced by ⋀^{2} E ≃ K_{C}, and the residue map r : K_{C}(2D)/K_{C} ⟶ C given by

r(ω) = ∑_{P ∈Supp D} Res_{P} ω.

This induces a non-degenerate skew-symmetric pairing ⟨ , ⟩ on the vector space H^{0}(A) of dimension 4N. Since r is identically zero on the image of H^{0}(K_{C}(2D)) (by the Residue Theorem), so is ⟨ , ⟩ on the image V of H^{0}(E(D)) ⟶ H^{0}(A). By the exact sequence

0 ⟶ E(−D) ⟶ E(D) ⟶ A ⟶ 0,

the image V is isomorphic to the quotient space H^{0}(E(D))/H^{0}(E(−D)). Hence V is a Lagrangian of the symplectic vector space H^{0}(A) by (4.5). It is obvious that V_{0} = H^{0}(E/E(−D)) is also a Lagrangian. Now we choose D so that H^{0}(E(−D)) = 0 for every [E] ∈ M_{C}(2, K). Then H^{0}(E) is the intersection of two Lagrangians V = H^{0}(E(D)) and V_{0} of H^{0}(A). Hence we have our inequality by the above proposition. □

Remark 4.6. This proof works also for a vector bundle E of even rank with a non-degenerate skew-symmetric pairing E ×E ⟶ K_{C}. The same trick was used in [12] to show the parity preservation of h^{0}(E) when E has a non-degenerate quadratic form q : E ⟶ K_{C}.

Let E be a rank-two vector bundle on C. For f ∈ Hom(H^{0}(E), H^{1}(E)), let T (f) be the linear map

⋀^{2} H^{0}(E) ∋ h_{1} ∧ h_{2} ↦ h_{1}^{∪}f(h_{2}) − h_{2}^{∪}f(h_{1}) ∈ H^{1}(⋀^{2} E),

where ^{∪} : H^{0}(E) × H^{1}(E) ⟶ H^{1}(⋀^{2} E) is the cup product. It is easy to check that the following diagram is commutative:

H^{1}(End E) ⟶  Hom(H^{0}(E), H^{1}(E))
H^{1}(tr) ↓  ↓ T
H^{1}(O_{C}) ⟶ Hom(⋀^{2} H^{0}(E), H^{1}(⋀^{2} E)),

(4.7)

where the lower horizontal linear map is the composite of

τ_{det E} : H^{1}(O_{C}) ⟶ Hom(H^{0}(⋀^{2} E), H^{1}(⋀^{2} E)),

defined in Section 2, and Hom(λ_{E}, H^{1}(⋀^{2} E)).

Theorem 4.8. Assume that [E] ∈ M_{C}(2, K, n) and h^{0}(E) = n + 2. Then:

(1) the Zariski cotangent space of M_{C}(2, K, n) at [E] is isomorphic to the cokernel of S^{2}H^{0}(E) ⟶ H^{0}(S^{2}E),

(2) dim Coker[S^{2}H^{0}(E) ⟶ H^{0}(S^{2}E)] ≥ dim_{[E]} M_{C}(2, K, n) ≥ σ(E), and

(3) M_{C}(2, K, n) is smooth and of dimension σ(E) at [E] if and only if the map S^{2}H^{0}(E) ⟶ H^{0}(S^{2}E) is injective.

Proof. As we saw in Section 3, the tangent space of M_{C}(2, K) at [E] is the kernel H^{1}(sl(E)) of the trace map H^{1}(End E) ⟶ H^{1}(O_{C}). Let Ẽ be a first-order infinitesimal deformation of E corresponding to B = {B_{ij}} ∈ H^{1}(End E). By the same argument as in the proof of Proposition 2.6, a global section h ∈ H^{0}(E) extends that of Ẽ if and only if the cup product h^{∪}B ∈ H^{1}(E) vanishes. Hence, all global sections of E extend if and only if the cup product map^{∪}B : H^{0}(E) ⟶ H^{1}(E) is zero. Therefore, the Zariski tangent space of M_{C}(2, K, n) is the kernel of H^{1}(sl(E)) ⟶ Ker T ⊂ Hom(H^{0}(E), H^{1}(E)). Since ⋀^{2} E ≃ K_{C}, the dual of (4.7) is reduced to the commutative diagram

H^{0}(E ⊗ E) ⟵ H^{0}(E) ⊗ H^{0}(E)
↑  ↑
H^{0}(⋀^{2} E) ⟵^{λ_{E}}  ⋀^{2} H^{0}(E),

by Serre duality (3.3), which shows (1). Part (2) follows from (1) and Theorem 4.2. Part (3) is straightforward from (1) and (2). □

Remark 4.9. The obstructions for M_{C}(2, K, n) to be smooth at [E] lie in the cokernel of H^{0}(S^{2}E)^{∨} ⟶ S^{2}H^{0}(E)^{∨}. This fact gives another proof of Theorem 4.2 and 4.8.

Let σ be the right-hand side of Theorem 4.2. Theorem 2.7, 2.8 and 2.9 lead us to the following problems: Problem 4.10.

(Existence) Is \overline{M}_{C}(2, K, n) non-empty when σ ≥ 0?
(Connectedness) Is \overline{M}_{C}(2, K, n) connected when σ > 0?

Problem 4.11. Assume that [C] ∈ M_{g} is general.

1) Is S^{2}H^{0}(E) ⟶ H^{0}(S^{2}E) injective for every E ∈ M_{C}(2, K)?

2) Is M_{C}(2, K, n) of dimension σ when σ ≥ 0?

3) Compute the number of M_{C}(2, K, n) when σ = 0. More generally, describe the cohomology class of \overline{M}_{C}(2, K, n) in H^{∗}(\overline{M}_{C}(2, K), Z) when σ > 0. Another direction is

Problem 4.12. Study the Grassmannian map associated with a member of M_{C}(2, K, n), and its relation with the canonical model C_{2g−2} ⊂ P^{g−1}.

We give some sample results in these directions. They are closely related to our classification of Fano threefolds via vector bundles [8]. We first consider the three cases (g, n + 2) = (7, 5), (9, 6) and (11, 7). The Brill–Noether number σ is equal to 3, 3 and 2, respectively.

Theorem 4.13. Let C be a curve of genus 7 with W_{4}^{1}(C) = ∅. Then:

1) M_{C}(2, K, 3) is smooth, complete and of dimension 3;

2) M_{C}(2, K, 3) is a Fano threefold of genus 7, i.e., (−K_{M})^{3} = 12, and with Picard number one; and

3) the intermediate Jacobian H^{2}(Ω_{M}^{1})/H^{3}(M, Z) of M_{C}(2, K, 3) is isomorphic to the Jacobian of C as polarized abelian variety. Conversely, every smooth Fano threefold of genus 7 with Picard number one

is obtained in this manner from a non-tetragonal curve of genus 7.

Similarly, \overline{M}_{C}(2, K, 4) is a quartic threefold in P^{4} with 21 singular points at the boundary if C is a general curve of genus 9, and M_{C}(2, K, 5) is a (polarized) K3 surface of genus 11 if C is a general curve of genus 11.

In the case (g, n + 2) = (8, 6), the number σ is equal to zero.

Theorem 4.14 (Mukai [9], [10]). If C is a curve of genus 8 with W_{7}^{2}(C) = ∅, then M_{C}(2, K, 4) consists of the unique isomorphism class of stable two-bundles E. The linear map λ_{E} in (3.1) is surjective and the following diagram, essentially (3.2), is Cartesian:

C  ⟶^{Φ_{|E|}} G(H^{0}(E), 2)
canonical embedding ∩  ∩ Plücker embedding
P^{∗}H^{0}(K_{C}) ⟶^{P^{∗}λ_{E}} P^{∗} ⋀^{2} H^{0}(E).

In particular, C is a complete linear section of the 8-dimensional Grassmannian, that is,

[C ⊂ P^{7}] = [G(6, 2) ⊂ P^{14}] ∩ H_{1} ∩ ⋯ ∩ H_{7}

for seven hyperplanes H_{1}, …, H_{7}.

Let C and E be as in the theorem and consider the intersection of G(2, H^{0}(E)) and P_{∗} Ker λ_{E}, where G(2, H^{0}(E)) is the Grassmannian of two-dimensional subspaces of H^{0}(E) embedded into P_{∗} ⋀^{2} H^{0}(E) by the Plücker coordinates. If a subspace [U] ∈ G(2, H^{0}(E)) belongs to P_{∗} Ker λ_{E}, the evaluation homomorphism ev^{U} : U ⊗ O_{C} ⟶ E is not injective and its kernel is a line bundle. Moreover, the inverse of Ker ev^{U} belongs to W_{5}^{1}(C), and if C is general, the map

(4.15) G(2, H^{0}(E)) ∩ P_{∗} Ker λ_{E} ⟶ W_{5}^{1}(C), [U] ↦ (Ker ev^{U})^{−1}

is an isomorphism between two reduced zero-dimensional schemes, which shows Theorem 2.9 in the case (a, b) = (2, 4). This idea leads us to a computation-free proof of Theorem 2.9, which we will discuss elsewhere.

References

1. E. Arbarello, M. Cornalba, P. A. Griffiths, and J. Harris, Geometry of Algebraic Curves I, Springer, New York, 1985.

2. A. Beauville, Variétés de Prym et jacobiennes intermédiaires, Ann. Sci. École Norm. Sup. (4), 10 (1977), 309–391.

3. S. K. Donaldson, A new proof of a theorem of Narasimhan and Seshadri, J. Diff. Geom. 18 (1983), 269–2277.

4. D. Gieseker, On the moduli of vector bundles on an algebraic surface, Ann. of Math. 106 (1977), 45–60.

5. , Stable curves and special divisors: Petri’s conjecture, Invent. Math., 66 (1982), 251–275.

6. R. Lazarsfeld, A sampling of vector bundle techniques in the study of linear system, Lectures on Riemann surfaces, Trieste, 1987 (M. Cornalba et al., eds.), World Scientific, Singapore, 1989, pp. 500–559.

7. , Brill–Noether–Petri without degenerations, J. Diff. Geom. 23 (1986), 299–307.

8. S. Mukai, Fano 3-folds, Complex Projective Geometry (G. Ellingsrud, ed.), London Math. Soc. Lecture Note Series 179, Cambridge University Press, 1992, pp. 255–263.

9. , Curves and symmetric spaces, Proc. Japan Acad. 68 (1992), 7–10.

10. , Curves and Grassmannians, Algebraic Geometry and Related Topics, Inchon, Korea, 1992, International Press, Boston, 1993, pp. 19–40.

11. D. Mumford, Projective invariants of projective structures and applications, Internat. Cong. Math. Stockholm, 1962, pp. 526–530.

12. , Theta characteristic of an algebraic curve, Ann. Sci. École Norm. Sup. (4), 4 (1971), 181–192.

13. , Curves and their Jacobians, The University of Michigan Press, 1975.

14. and J. Fogarty, Geometric Invariant Theory, second edition, Springer, 1982.

15. and P. E. Newstead, Periods of a moduli space of bundles on a curve, Amer. J. Math. 90 (1968), 1201–1208.

16. M. S. Narasimhan and Seshadri C. S., Stable and unitary vector bundles on a compact Riemann surface, Ann. of Math. 82 (1965), 540–564.

17. P. E. Newstead, Stable bundles of rank 2 and odd degree over a curve of genus 2 , Topology 7 (1968), 205–215.

18. , Brill–Noether Problems List Update, University of Liverpool, 1992.

19. S. Ramanan, The moduli space of vector bundles over an algebraic curve, Math. Ann. 200 (1973), 69–84.

20. M. Teixidor, Brill–Noether theory for stable vector bundles, Duke Math. J., 62 (1991), 385–400.

21. I. V. Desale and S. Ramanan, Classification of vector bundles of rank 2 on nyperelliptic curves, Invent. Math., 38 (1976), 161–185.

Added in Proof. Our determinantal description of M_{C}(2, K, n) is similar to that of the loci of special divisors in Prym varieties, for which Problems 4.10 and 4.11 have already been solved. See:

22. G. E. Welters, A theorem of Gieseker–Petri type for Prym varieties, Ann. Sci. École Norm. Sup. (4), 18 (1985), 671–683.

23. A. Bertram, An existence theorem for Prym special divisors, Invent. Math. 90 (1987), 669–671.

24. C. de Concini and P. Pragacz, On the class of Brill–Noether loci for Prym varieties, preprint.

Shigeru Mukai

Department of Mathematics, School of Science Nagoya University 464-01 Furō-chō, Chikusa-ku Nagoya, Japan E-mail address: mukai@math.nagoya-u.ac.jp