HYPOELLIPTICITY: GEOMETRIZATION AND SPECULATION

MICHAEL CHRIST

Abstract. To any finite collection of smooth real vector fields X_{j} in ℝ^{n} we associate a metric in the phase space T^{∗}ℝ^{n}. The relation between the asymptotic behavior of this metric and hypoellipticity of ∑ X_{j}^{2}, in the smooth, real analytic, and Gevrey categories, is explored.

To Professor P. Lelong, on the occasion of his 85th birthday, in admiration.

1. Introduction

Let {X_{j}} be a collection of real vector fields with C^{∞} or C^{ω} coefficients, defined in a neighborhood of a point x_{0} ∈ ℝ^{d}. Consider a second order differential operator L = ∑_{j} X_{j}^{2}. Under what conditions is L hypoelliptic, in C^{∞} or C^{ω}? As is well known, certain of these sums of squares operators provide a good model for aspects of the ¯∂– Neumann problem in ℂ^{2}, and a grossly oversimplified but still illuminating window into the higher dimensional case. Indeed, application of the method of boundary reduction to the ¯∂–Neumann problem leads to a pseudodifferential equation on the boundary; microlocalizing and composing with an elliptic factor then leads in ℂ^{2} to an operator closely related to X^{2} + Y^{2} for certain vector fields. We believe that the study of more general collections of vector fields than those directly relevant to the ¯∂–Neumann has shed, and will shed, light on the special cases of interest in complex analysis.

In a companion paper [7] we have established a sufficient condition for the C^{∞} case which subsumes a variety of previously known results and examples. This condition involves the existence of auxiliary operators, having certain favorable commutation relations with L.

Except in very special cases, there remains a large gap between the hypotheses of the sufficient conditions for hypoellipticity in various function spaces, and counterexamples suggesting what hypotheses are necessary. Many works on various types of hypoellipticity and regularity have relied on commutation methods to achieve results in the positive direction, whereas negative results have been attained through different methods.

How closely is hypoellipticity linked with the commutation method? The aim of this note is to begin to directly investigate this question. We will:

1. Define a metric ρ_{L}, adapted to L, on the cotangent bundle, and show how a necessary condition for the existence of auxiliary operators having desirable commutation properties with L may be formulated in terms of ρ_{L}.

Research at MSRI is supported in part by NSF grant DMS-9701755. Research also supported by NSF grant DMS-9623007.

2. Speculate on the connection between properties of ρ_{L} and hypoellipticity of L in C^{ω}, in Gevrey classes, and to a lesser extent in C^{∞}.

3. Analyze the behavior of ρ_{L} in fundamental examples.

A well known construction associates to the collection {X_{j}} of vector fields a Carnot-Caratheodory, or control, metric on the base space ℝ^{d}, rather than on the cotangent bundle. This metric, and its connection with the fundamental solution of ∑ X_{j}^{2}, has been intensively studied by Fefferman and Phong [12], by Nagel, Stein and Wainger [17], and by Sánchez-Calle [19]. A cotangent bundle metric has been defined by Fefferman and Parmeggiani [11],[18]. Their metric differs from ρ_{L} in fundamental respects, as explained in §4 below.

2. Definitions

Let μ : T^{∗}ℝ^{d} ↦ (0, ∞) be any positive nonvanishing real symbol belonging to the class S_{1,0}^{1}, and let {V_{j}} be any finite collection of vector fields, defined on a conic open subset of the cotangent bundle T^{∗}ℝ^{d}, with real C^{∞} coefficients. Assume that each V_{j} takes the form ∑_{k} a_{j,k}∂/∂x_{j} + ∑_{k} b_{j,k}∂/∂ξ_{j} where a_{j,k} ∈ S_{1,0}^{0} and b_{j,k} ∈ S_{1,0}^{1}.

Our first definition is made only to facilitate the main definitions below.

Definition 1. A locally C^{1} function ψ : T^{∗}ℝ^{d} ↦ ℝ is said to be Lipschitz relative to (μ, {V_{j}}) if at every point p = (x, ξ) of T^{∗}ℝ^{d},

|V_{j}ψ(p)| ≤ μ(p) for all j ≤ n (1)

and

μ(p)⟨ξ⟩^{−1}|∇_{x}ψ(p)| + μ(p)|∇_{ξ}ψ(p)| ≤ μ(p). (2)

The main condition here is (1), which imposes a stringent upper bound on the variation of ψ in the directions V_{j}, at places where μ(x, ξ) is smaller than ⟨ξ⟩ and some vector field V_{j} is relatively large. The common factor of μ(p) on both sides of (2) is not a typographical error and will be explained below.

Next, let finitely many C^{∞} real vector fields X_{j} be given on an open subset of the base space ℝ^{d}. Recall that to any smooth function f : T^{∗}ℝ^{d} ↦ ℝ is associated the Hamiltonian vector field

H_{f} = ∑_{n=1}^{d} (\frac{∂f}{∂x_{n}} \frac{∂}{∂ξ_{n}} − \frac{∂f}{∂ξ_{n}} \frac{∂}{∂x_{n}} ) .

Define σ_{j}(x, ξ) to be the principal symbol of iX_{j}, which is real valued. Denote by H_{σ_{j}} the Hamilton vector field associated to σ_{j}.

In most of the discussion we assume the bracket hypothesis to be satisfied, to some order m. Then consider the set S consisting of all multi-indices I = (I_{1}, … I_{k}) of degree |I| = k ∈ {1, 2, … m} such that each I_{s} ∈ {1, 2, … n}. For each I ∈ S define σ_{I}(x, ξ) to be the iterated Poisson bracket {σ_{I_{k}}, σ_{I′}} where I′ = (I_{1}, … I_{k−1}). Define the effective symbol^{1} σ̃ of L to be

σ̃(x, ξ)^{2} = ∑_{I∈S} |σ_{I}(x, ξ)|^{2/|I|}. (3)

The bracket hypothesis is equivalent to the existence of C, ε ∈ ℝ^{+} and m such that

σ̃(x, ξ) ≥ C⟨ξ⟩^{ε} for all (x, ξ).

The symplectic structure of the cotangent bundle is reflected in the next definition.

Definition 2. A C^{1} function ψ defined on T^{∗}ℝ^{d} is said to be microlocally Lipschitz relative to {X_{j}} if it is Lipschitz relative to (μ, {V_{j}}) = (σ̃, {H_{σ_{j}}}) where σ̃ is the effective symbol defined in (3), and H_{σ_{j}} is the Hamiltonian vector field associated to the principal symbol σ_{j} of iX_{j}.

With this definition, even when L = ∑ X_{j}^{2} is elliptic, (1) by itself does not force ψ even to be continuous; when L has constant coefficients, for instance, each V_{j} = H_{σ_{j}} is everywhere in the span of {∂/∂x_{l}}. The auxiliary condition (2), which does not depend on the vector fields X_{j} at all because the same factor of μ occurs on both sides, does ensure that ψ is locally Lipschitz with respect to the standard metric

dρ_{0}^{2} = ⟨ξ⟩^{2}dx^{2} + dξ^{2}.

(2) is simply one of the inequalities defining the symbol class S_{1,0}^{1}, rewritten so as to more closely resemble (1). (2) implies (1) when L is elliptic, but not in general.

Our main definition is as follows. Associated to {X_{j}}, via σ̃ and the collection of Hamiltonian vector fields {H_{σ_{j}}}, is a metric ρ on T^{∗}ℝ^{d}.

Definition 3. Let X_{j} be real C^{∞} vector fields in an open subset of ℝ^{d}. Then

ρ_{{X_{j}}}(p, q) = sup_{ψ} |ψ(p) − ψ(q)|, (4)

where the supremum is taken over all functions ψ that are microlocally Lipschitz relative to {X_{j}}. To L = ∑ X_{j}^{2} is associated the metric

ρ_{L} = ρ_{{X_{j}}}. (5)

The motivation for this definition stems from a proof of positive hypoellipticity results; see [7].

ρ_{L} is defined as a metric in the sense of point set topology, but the definition could be modified to give a Riemannian metric whose properties are equivalent for our purpose. (2) implies that for any collection of vector fields X_{j}, there exists C < ∞ such that ρ_{{X_{j}}} ≤ Cρ_{0}.

One property of ρ_{L} is that L is elliptic if and only if ρ_{L} is comparable (on the infinitesimal level) to ρ_{0}.

1In the absence of the bracket hypothesis, there is no satisfactory general description of the effective symbol. Lemma 3.1 of [7] describes one narrow class of nonsubelliptic operators for which there exists an effective symbol which is significantly larger than the principal symbol.

Remark. The construction of ρ_{L} is conformally invariant in the sense that if L_{2} = r^{2} · L_{1} for some constant r ≠ 0, and if L_{1} is a subelliptic sum of squares operator, then ρ_{1} ≡ ρ_{2}.

If L_{1} = ∑_{j} X_{j}^{2} and L_{2} = ∑_{j}(r(x)X_{j})^{2} where 0 ≠ r ∈ C^{∞}, then the two metrics have the same asymptotic behavior, in the sense that their ratio tends to one as the ξ coordinate tends to infinity through any connected conic open set in which L_{1} has strictly subelliptic behavior in the sense that C^{−1}ρ_{0}^{δ} ≤ ρ_{1} ≤ Cρ_{0}^{1−δ} for separated pairs of points.

There exists another construction which roughly speaking is intermediate between ρ_{L} and the more familiar Carnot-Caratheodory metric, and which will help us to illustrate what phenomenon is measured by ρ_{L} but not seen by the Carnot-Caratheodory metric. To any finite family {X_{j}} of smooth real vector fields on a connected open set U ⊂ ℝ^{d}, this construction associates a one parameter family ϱ_{R} of metrics on U, rather than a single metric on phase space.

Define the auxiliary quantity ν : U × ℝ^{+} ↦ ℝ^{+} by

ν(x, R) = min_{|ξ|=R} σ̃(x, ξ). (6)

One property of ν is that for any R, R′ ≥ 1 satisfying R ∼ R′, one has ν(x, R) ∼ ν(x, R′), uniformly in x, R, R′. If the bracket condition holds to order m at x, then ν(x, R) ≥ cR^{1/m} as R → ∞.

Definition 4. ϱ_{R}(p, q) is the supremum of |ψ(p) − ψ(q)|, taken over all functions ψ satisfying

|X_{j}(ψ)(x)| ≤ ν(x, R) for all j (7)

and

ν(x, R)R^{−1}|∇_{x}ψ| ≤ ν(x, R). (8)

An equivalent formulation is that ϱ_{R} is the Carnot-Caratheodory metric associated to {ν(x, R)^{−1}X_{j}} ∪ {R^{−1}∂/∂x_{k} : 1 ≤ k ≤ d}. If the bracket condition holds to order m at every point of U, then ϱ_{R}(·, ·) ≥ cR^{1/m} on any compact subset of U × U minus the diagonal, as R → ∞.

This construction takes a simpler form in the important special case where the set of vector fields X_{j} has cardinality d − 1 and is everywhere linearly independent. Then the characteristic variety of ∑ X_{j}^{2} is a line bundle. It may be easily checked that a quantity equivalent to ν(x, R) is obtained by deleting the supremum in (6) and instead taking ξ to be an element of the fiber of that bundle lying over x, of magnitude R.

3. The Question

The following question may be fundamental. Let s ∈ [1, ∞). We say that two sets Γ, Γ′ ⊂ T^{∗}ℝ^{d} are c–separated if any two points p = (x, ξ) ∈ Γ, p′ = (x′, ξ′) ∈ Γ′ satisfy |x − x′| + (1 + |ξ| + |ξ′|)^{−1}|ξ − ξ′| > c. Let {X_{j}} be a finite collection of C^{ω} real vector fields.

Main Question. Consider the condition: For every c > 0, there exists c′ > 0 such that for any two c–separated open sets Γ, Γ′ ⊂ T^{∗}ℝ^{d},

(9)  ρ_{L}(p, p′) ≥ c′ρ_{0}(p, p′)^{1/s}  for all p ∈ Γ, p′ ∈ Γ′.

Is (9) or some closely related condition nearly equivalent to microlocal hypoellipticity of ∑ X_{j}^{2} in the Gevrey class G^{s}?

It is essential in (9) that Γ, Γ′ be separated; thus no infinitesimal comparison of metrics is made.^{2} Restricting attention to pairs (p, p′) for which ρ_{0}(p, p′) ∼ 1, rather than merely ≳ 1, results in an equivalent question.

In the remainder of this paper, the metric ρ_{L} will be analyzed for a number of examples, the optimal C^{∞}, analytic, and Gevrey class hypoellipticity properties of which are already known for each example but the last. In each known case, the result will be seen to be consistent with an affirmative answer to this question, with the word “nearly” omitted.

Nonetheless we have deliberately refrained from posing the question more precisely or labeling it a conjecture. Some variant of (9) may be more directly related to hypoellipticity. Some such variants impose stronger restrictions on ψ, involving upper bounds on its partial derivatives of orders greater than one, requiring it to belong to an appropriate symbol class; thus when s = 1, ψ should perhaps be an analytic symbol as defined for instance in [21]. Our question merely captures a first order approximation to such a condition. Such variants of (9) are genuinely inequivalent, as shown in Example 9 below. To the extent of this author’s knowledge, such a stronger variant is indeed satisfied with s = 1 for every operator that has so far been proved to be analytic hypoelliptic.

As will be seen in the subsequent discussion, (9) for s = 1 is related to a conjectured criterion for analytic hypoellipticity proposed by Treves [22]. It would be worthwhile to further analyze this connection. Our final Example 9 is an interesting test case which may distinguish between the two points of view.

For C^{∞} hypoellipticity the analogous condition would be that for every A < ∞ there should exist C_{A} < ∞ such that

ρ_{L}(p, p′) ≥ A log ρ_{0}(p, p′) − C_{A} (10)

for all p ∈ Γ, p′ ∈ Γ′, where ρ_{L} would denote the metric associated to an appropriately defined effective symbol of ∑ X_{j}^{2}, if a meaningful effective symbol were to exist.

4. Examples

The notation A ∼ B will mean that the quantity A/B is bounded above and below by positive finite constants, uniformly in all relevant parameters; usually these parameters will be points in the cotangent bundle or certain subregions thereof, and a large parameter λ. Throughout the discussion, the zero section is always understood to be excluded from any cotangent bundle.

2When s = 1, the effective symbol may be replaced by (1 + ∑_{j} σ_{j}^{2})^{1/2} in the definition of ρ_{L} without affecting the validity of (9).

Example 1. Let L = ∑ X_{k}^{2} be an arbitrary sum of squares operator satisfying the bracket hypothesis to order m at every point of an open set U ⊂ ℝ^{d}. Then σ̃(x, ξ) ≥ c(1 + |ξ|)^{1/m}. Therefore in the region where |ξ| ∼ λ, the functions λ^{1/m}x_{j} and λ^{−(m−1)/m}ξ_{j} are microlocally Lipschitz, up to a constant factor, uniformly for large λ.

From this it follows that

ρ_{L}(p, q) ≥ cρ_{0}(p, q)^{1/m} (11)

whenever ρ_{0}(p, q) ≥ 1. This is consistent with an affirmative answer to our main question, a theorem of Derridj and Zuily asserting that such operators are Gevrey class hypoelliptic of every order s ≥ m, and the existence of such operators which are not Gevrey hypoelliptic of any smaller order, such as (13).

On the other hand, for any operator we have ρ_{L} ≤ Cρ_{0}. In particular, when L is elliptic, ρ_{L} is pointwise comparable to ρ_{0}.

Remark. This conclusion for elliptic operators underscores the contrast between ρ_{L} and a metric ρ̃_{L} introduced by Fefferman and Parmeggiani [11],[18]. Firstly, the correspondence between L and ρ̃_{L} has a fundamental monotonicity property: For any two operators L_{1} and L_{2} whose principal symbols are nonnegative and satisfy σ_{1}(p) ≤ σ_{2}(p) for every point p ∈ T^{∗}ℝ^{n}, one has ρ̃_{L_{1}}(p, p′) ≥ ρ̃_{L_{2}}(p, p′) for all p, p′. The mapping from L to ρ_{L} has by design no such monotonicity; this reflects the failure of hypoellipticity of L_{1} (in various function spaces) to imply hypoellipticity of L_{2} when σ_{1} ≤ σ_{2}, in general. The Carnot-Caratheodory metric associated to L on the base space ℝ^{d} likewise shares this monotonic dependence on L.

Secondly, up to order of magnitude, ρ_{L} is largest when L is elliptic, whereas ρ̃_{L} is then smallest.

Example 2. The simplest type of obstruction to (9) in the case s = 1 is the possible existence of a nonvanishing vector field V in the span of {H_{σ_{j}}}, such that some integral curve γ of V is contained in the characteristic variety Σ = {p : σ_{j}(p) = 0 for all j}. Treves [20] has conjectured that this obstruction should preclude analytic hypoellipticity of L.

To see that (9) must fail to hold with s = 1 in this situation, let p = (x, ξ) and p′ = (x′, ξ′) be any two distinct points of γ, and for large λ ∈ ℝ^{+} set λp = (x, λξ) and λp′ = (x′, λξ′). Consider the integral curve γ_{λ} joining λp to λp′, parametrized by s ↦ exp(sV)(λp); we have λp′ = exp(c_{0}V)(λp), where c_{0} is a positive constant independent of λ, because the symbols σ_{j} of the underlying vector fields are homogeneous functions of degree one on T^{∗}ℝ^{d}. Moreover γ_{λ} is contained in Σ by hypothesis. As is well known, this implies a loss of at least one half of a derivative:

σ̃(x, ξ) ≤ C(1 + |ξ|)^{1/2} ∼ λ^{1/2}

along γ_{λ}.

Any microlocally Lipschitz function ψ must satisfy |V (ψ)| ≤ σ̃ ≤ Cλ^{1/2} along γ_{λ}. Consequently |ψ(λp) − ψ(λq)| ≤ Cλ^{1/2}. Taking the supremum over all such ψ yields

ρ_{{X_{j}}}(λp, λp′) ≤ Cρ_{0}(λp, λp′)^{1/2}

as λ → +∞. Consequently (9) fails to be satisfied for s = 1.

More precisely, if σ̃(λq) ≤ Cλ^{1/m} for every q belong to γ_{λ}, then

ρ_{L}(λp, λp′) ≤ Cρ_{O}(λp, λp′)^{1/m}. (12)

A concrete example is the Baouendi-Goulaouic type operator [1] in ℝ^{3}:

∂_{x}^{2} + ∂_{y}^{2} + (x^{m−1}∂_{t})^{2} (13)

Take p = (0, 0, 0; 0, 0, 1) and p′ = (0, 1, 0; 0, 0, 1) and V = ∂_{y}.

Remark. There are two different ways to formulate the conjecture of Treves mentioned in the discussion of the preceding example. One involves curves γ ⊂ Σ whose tangent vectors γ̇ belong to the span of {H_{σ_{j}}} at each point of γ; the other involves curves for which γ̇ is orthogonal, with respect to the skew symmetric form defined by the canonical symplectic form ω, to the tangent space to Σ at each point of γ. These two conditions on γ are in fact equivalent, under the assumption that the collection of vector fields ∇_{x,ξ}σ_{j} has constant rank in a neighborhood of p = γ(0).

Indeed, Σ is by assumption a smooth manifold in a neighborhood of p whose tangent space is the set of all vectors v ∈ T_{p}(T^{∗}ℝ^{d}) satisfying v(σ_{j}) = 0 for all j. Denote by W^{⊥} the orthocomplement, with respect to ω, of any subspace W of T_{p}Σ. Since ω is nondegenerate, (W^{⊥})^{⊥} = W for all subspaces. Define V to be the span of {H_{σ_{j}}(p)}, a linear subspace of T_{p}Σ.

Let w = γ̇(0) ∈ T_{p}Σ. Then w ⊥_{ω} T_{p}Σ if and only if ω(v, w) = 0 for all v such that v(σ_{j}) = 0 for all j. Since v(σ_{j}) = ω(v, H_{σ_{j}}), the latter condition on w may be restated as ω(v, w) = 0 for all v ∈ V^{⊥}. Thus w ⊥_{ω} T_{p}Σ if and only if w ∈ (V^{⊥})^{⊥} = V.

Remark. A more general conjecture formulated by Treves in [22] involves the construction of a stratification Σ = Σ_{0} ⊃ Σ_{1} ⊃ ⋯, based upon the vanishing not only of the symbols σ_{j}, but also of their iterated Poisson brackets, as well as on quantities such as the dimension of the span of their gradients and the rank of the quadratic form defined by the restriction to the tangent space of a stratum of the symplectic form. One considers curves γ ⊃ Σ_{k}\Σ_{k+1} whose tangents are orthogonal to T Σ_{k}, with respect to ω. According to the preceding remark, this means that γ̇ belongs to the span of the set of Hamiltonian vector fields associated to the collection of all iterated Poisson brackets of the symbols σ_{j} up to a certain order n, depending on k. The discussion of the preceding example may be generalized to conclude that the existence of such a curve γ implies a certain upper bound on ρ_{L}, depending on n.

Example 3. Suppose that X_{1}, … X_{d−1} are linearly independent at each point of U ⊂ ℝ^{d}, which throughout this example are assumed to satisfy the bracket hypothesis to order two at every point of U. Then ρ_{L}(p, q) ≥ cρ_{0}(p, q)^{1/2} for separated points, as we have seen in Example 1. In Example 2 it is shown that this lower bound cannot in general be improved.

If however the characteristic variety Σ = {(x, ξ) : σ_{j}(x, ξ) = 0 for all j} is a symplectic manifold, then each Hamiltonian vector field H_{σ_{j}} is everywhere transverse to Σ. From this is easily deduced the improvement ρ_{L}(p, q) ≥ cρ_{0}(p, q)^{1} for all separated points. This is consistent with the fundamental theorem of Tartakoff and of Treves, which asserts analytic hypoellipticity in the symplectic situation.

Now consider the one parameter family of metrics ϱ_{R} on U of Definition 4. Σ is a line bundle over U, and ν(x, R) ∼ R^{1/2} for all x, R. From this and the order two bracket hypothesis it is easily deduced that ϱ_{R}(x, y) is comparable to R^{1/2} times the associated Carnot-Caratheodory metric at distances greater than R^{−1/2} with respect to the latter metric. In particular, ϱ_{R}(x, y) ∼ R^{1/2} for fixed x ≠ y. This is so, regardless of whether Σ is symplectic. Thus the family ϱ_{R} fails to detect a property of the vector fields which is fundamental for analytic hypoellipticity.

Example 4. Let W ⊂ ℝ^{d} be any connected open set. Let Γ be a conic open subset of T^{∗}W. Suppose that L is elliptic in Γ. Then for separated points p, q lying in distinct topological components of T^{∗}W \Γ, ρ_{L}(p, q) ≥ cρ_{0}(p, q). Indeed, any ψ ∈ S_{1,0}^{1} whose gradient is supported in Γ is microlocally Lipschitz, up to multiplication by a bounded constant factor, and there exist such functions satisfying |ψ(p) − ψ(q)| ∼ ρ_{0}(p, q).

More generally, if σ̃(x, ξ) ≥ C|ξ|^{δ} for all (x, ξ) ∈ Γ, then by the same reasoning ρ_{L}(p, q) ≥ cρ_{0}(p, q)^{δ} for all p, q satisfying the same hypothesis.

The next example concerns C^{∞} hypoellipticity. In the C^{∞} case one seeks only to have ρ(p, p′) ≫ log ρ_{0}(p, p′) for separated p, p′. Under the very weak hypothesis that L has an effective symbol σ̃(x, ξ) which tends to infinity as |ξ| → ∞, this bound is easily obtained whenever |ξ| and |ξ − ξ′| are large, because the function ψ(x, η) = log(|η − ξ|) is always locally Lipschitz modulo a constant factor, and an arbitrarily large constant multiple of ψ is Lipschitz relative to {X_{j}} for sufficiently large |ξ|, provided that σ̃ → ∞.

Example 5. In ℝ^{2} with coordinates (x, t) consider L = X^{2} + Y^{2} where

X = ∂_{x}, Y = a(x)∂_{t}.

Suppose that a ∈ C^{∞}, and that a(x) = 0 if and only if x = 0. All such operators are C^{∞} hypoelliptic [9], regardless of their degree of degeneracy. Let us see that this is consistent with our point of view.

The characteristic variety is Σ = {x = ξ = 0} (which is symplectic). Because of the remark made two paragraphs above, the condition ρ_{L}(p, q) ≫ log ρ_{0}(p, q) is most likely to fail when p, q ∈ Σ and both have the same τ coordinate. Thus let p = (0, 0, 0, τ) and q = (0, 0, 1, τ) with τ large and positive. Consider ψ(x, t, ξ, τ) = tτ. Certainly |ψ(p)−ψ(q)| = τ is ≫ log ρ_{0}(p, q) ∼ log τ, and we claim that ψ is Lipschitz. Indeed, the two Hamiltonian vector fields are ∂_{x} and a(x)∂_{t} − τa′(x)∂_{ξ}. The former annihilates ψ, while applying the latter to ψ gives a(x)τ, which equals the symbol of Y and hence is acceptably bounded, without taking into account the existence of an effective symbol slightly larger than |ξ| + |a(x)τ|.

Example 6. Let (x, t) be coordinates in ℝ^{2} and (x, t; ξ, τ) be coordinates in the cotangent bundle. Consider

(14)  L_{1} = ∂_{x}^{2} + (x∂_{t})^{2}
(15)  L_{2} = ∂_{x}^{2} + (x∂_{t})^{2} + (t∂_{t})^{2}.

The associated effective symbols are up to order of magnitude equal to σ̃_{1} ∼ 1 + |ξ| + |xτ| + |τ|^{1/2}, and σ̃_{2} ∼ 1 + |ξ| + |xτ| + |τ|^{1/2} + |tτ|. Thus σ̃_{1} ≤ Cσ̃_{2}.

The Hamiltonian vector fields associated to the vector fields comprising L_{1} are ∂_{x} and x∂_{t} − τ∂_{ξ}. For L_{2} there is the additional Hamiltonian field t∂_{t} − τ∂_{τ}. Thus there arise two competing effects in comparing ρ_{L_{2}} to ρ_{L_{1}}: σ̃_{2} is larger, but the presence of an additional Hamiltonian vector field introduces additional constraints on microlocally Lipschitz functions ψ.

To analyze the comparison, let λ ∈ ℝ^{+} be a large parameter and consider the two points p = (0, 0; 0, λ) and q = (0, 0; 0, 2λ). The function ψ(x, t; ξ, τ) = τ is microlocally Lipschitz with respect to the metric associated to L_{1}, because it belongs to S_{1,0}^{1} and is annihilated by the two Hamiltonian vector fields. Therefore ρ_{L_{1}}(p, q) ≥ cλ; it is comparable to the distance between these points in the metric associated to any elliptic operator, even though L fails to be elliptic.

Consider instead L_{2}, and consider the restriction of any microlocally Lipschitz function ψ to the line segment joining p to q. The Hamiltonian vector field t∂_{t} − τ∂_{τ} is tangent to this line segment, and σ̃_{2} has order of magnitude λ^{1/2} on the whole segment, so the main condition (1) forces |ψ(p) − ψ(q)| ≤ Cλ^{1/2}. Thus ρ_{L_{2}}(p, q) ≤ Cλ^{1/2} ≪ ρ_{L_{1}}(p, q). Indeed, Métivier [16] has proved that L_{2} is hypoelliptic in G^{s} only for s ≥ 2, whereas L_{1} is G^{s} hypoelliptic for all s ≥ 1 by a special case of a theorem of Grušin [14].

Further analysis shows that ρ_{L_{2}}(p, q) ≥ cρ_{0}(p, q)^{1/2} for any points p, q. L_{2} is known to be Gevrey hypoelliptic of order s if and only if s ≥ 2, so this example is consistent with an affirmative answer to our main question.

So far we have derived upper bounds on ρ_{L} by examining the variation of microlocally Lipschitz functions ψ along integral curves γ_{λ} ⊂ Σ of Hamiltonian vector fields. Our next two examples demonstrate subtler effects due to the influence of one parameter families of integral curves γ_{λ} which do not quite lie in the characteristic variety Σ, but merely tend to Σ at a certain rate as λ → ∞.

Example 7. With coordinates (x, y, t; ξ, η, τ) for T^{∗}ℝ^{3}, consider

∂_{x}^{2} + (x^{k−1}∂_{y})^{2} + (x^{m−1}∂_{t})^{2} (16)

where 2 ≤ k ≤ m. Here

σ̃ ∼ |ξ| + |x^{k−1}η| + |x^{m−1}τ| + |η|^{1/k} + |τ|^{1/m}.

Consider the associated Hamiltonian vector field V = x^{k−1}∂_{y} − (k − 1)x^{k−2}η∂_{ξ}. Let p = (δ, 0, 0; 0, 0, λ) where δ = λ^{−1/m}, and q = exp(T V)(p) where T = δ^{1−k}. The integral curve γ joining p to q takes the simple form exp(sV)(p) = (δ, sδ^{k−1}, 0; 0, 0, λ). In particular, q = (δ, 1, 0; 0, 0, λ). Along γ, σ̃ ∼ λ^{1/m} because δ was chosen so that δ^{k−1}λ ∼ λ^{1/m}. Thus any microlocally Lipschitz function ψ must satisfy |V ψ| ≤ Cλ^{1/m}, whence |ψ(p) − ψ(q)| ≤ CT λ^{1/m} = Cλ^{(k−1)/m}λ^{1/m} = λ^{k/m}.

On the other hand, the function ψ(x, y, t; ξ, η, τ) = λ^{k/m}y is microlocally Lipschitz in the region where |(ξ, η, τ)| ∼ λ, and |ψ(p) − ψ(q)| is comparable to λ^{k/m}. Thus

ρ_{L}(p, q) ∼ λ^{k/m} ∼ ρ_{0}(p, q)^{k/m}.

This is consistent with both an affirmative answer to our main question, and the known fact that L is hypoelliptic in the Gevrey class of order s if and only if s ≥ m/k, assuming that m ≥ k [4].

Example 8. Fix any integers m, r > 1. In ℝ^{2} consider

∂_{x}^{2} + (x^{m−1}∂_{t})^{2} + (t^{r}∂_{t})^{2}; (17)

the case r = 1 has been discussed above. One of the associated Hamiltonian vector fields is V = t^{r}∂_{t} −rt^{r−1}τ∂_{τ}. The effective symbol has order of magnitude |ξ|+|xτ|+ |t^{r}τ| + |τ|^{1/m}.

Given a large quantity λ, define δ = λ^{−(m−1)/mr}, so that δ^{r}λ = λ^{1/m}. Let p = (0, δ; 0, λ) and q = exp(−T V)(p) where T = δ^{1−r}. Denote by γ the segment of integral curve exp(−sV)(p), 0 ≤ s ≤ T. For every point of γ, uniformly in λ, exp(−sV)(p) = (0, t(s); 0, τ(s)) where δ ≥ t(s) ≥ cδ and λ ≤ τ(s) ≤ Cλ. Moreover q = (0, t(T); 0, τ(T)) where τ(T) − λ is positive and has order of magnitude λ. By the choice of δ and the fact that |t(s)| ≤ δ for every point of γ, σ̃ ≤ Cλ^{1/m} at every point of γ.

Every microlocally Lipschitz function ψ must therefore satisfy |V ψ| ≤ Cλ^{1/m} at every point of γ, and consequently |ψ(p) − ψ(q)| ≤ CT λ^{1/m}. Since T = δ^{1−r}, we conclude after a small computation that

ρ(p, q) ≤ Cλ^{(mr−m+1)/mr}. (18)

This upper bound on ρ_{L}(p, q) is also a lower bound, up to a constant factor. To see this consider the function ψ(x, t; ξ, τ) = λ^{a} · τ, where a + 1 = (mr − m + 1)/mr < 1. Then in the region where |(ξ, τ)| ∼ λ, ψ is a symbol in S_{1,0}^{1} uniformly in λ, and |V (ψ)| ≤ C|t|^{r−1}λ^{a+1}. To show that ψ is in this region microlocally Lipschitz relative to the vector fields in question, up to a bounded factor, we must verify that |V ψ| ≤ Cσ̃. Indeed, this holds because σ̃(x, t; ξ, τ) ≥ cλ^{1/m} + c|t|^{r}λ there, and |t|^{r−1}λ^{a+1} equals the logarithmically convex average (|t|^{r}λ)^{θ} · (λ^{1/m})^{1−θ} ≤ Cσ̃ where θ is defined to be (r − 1)/r. Thus ψ is microlocally Lipschitz, and

|ψ(p) − ψ(q)| ∼ λ^{(mr−m+1)/mr}, (19)

whence ρ_{L}(p, q) ≥ cλ^{(mr−m+1)/mr}. It has indeed been proved in [6], as well as in [2] and [15], that L is G^{s} hypoelliptic for all s satisfying s^{−1} ≤ (mr − m + 1)/mr. In unpublished work this author has outlined a proof that the threshold mr/(mr−m+1) is optimal, based on the method introduced in [3] and [5].

Remark. Both a conjectured characterization of analytic hypoellipticity by Treves [22], and a conjectured sufficient condition for Gevrey hypoellipticity by Bove and Tartakoff [2] are formulated in terms of properties of a certain stratification of the characteristic variety Σ of L. This stratification does not distinguish between different parameters r in Example 8; r does influence the optimal degree of Gevrey class hypoellipticity.

5. A Subtler Example

We next discuss an operator for which analytic hypoellipticity apparently remains an open question. It is the principal part of the Kohn Laplacian for a three dimensional pseudoconvex CR manifold, and is of particular interest both because it is one of the simplest operators whose analytic hypoellipticity is undecided, and because it illustrates the distinction between two variants of our main question; functions ψ which are merely Lipschitz relative to ρ_{L} can vary much more rapidly than those which also belong to the symbol class S_{1,0}^{1}.

Consider the domain ℑ(z_{2}) > b(z_{1}) in ℂ^{2}, where b is assumed to be subharmonic so that this domain will be pseudoconvex. Its boundary may be identified with ℝ^{3}, with coordinates (x, y, t), in such a way that a Cauchy-Riemann operator is ¯∂_{b} = X + iY where X = ∂_{x} − b_{y}∂_{t} and Y = ∂_{y} + b_{x}∂_{t}. Here b_{x} = ∂_{x}b, b_{y} = ∂_{y}b, and b(x, y) ≡ b(x + iy). The Kohn Laplacian is then (X + iY)(X − iY), which equals X^{2} + Y^{2} modulo a lower order term.^{3} Analytic or Gevrey hypoellipticity of such an operator depends only on Δb, where Δ denotes the Laplacian in ℝ^{2}, rather than on b itself, for any operator may be transformed into any other with the same invariant Δb via a change of variables (x, y, t) ↦ (x, y, t − ϕ(x, y)).

Example 9. In ℝ^{3} consider L = X^{2} + Y^{2} with

X = ∂_{x} − b_{y}∂_{t}, Y = ∂_{y} + b_{x}∂_{t}

where b is a polynomial depending only on x, y, satisfying

Δb(x, y) = x^{k} + y^{k} + x^{2}y^{2},

and k ≥ 6 is an integer. Define λ(x, y) = Δb = x^{k} + y^{k} + x^{2}y^{2}.

For general b(x, y) it has been shown by Grigis and Sjöstrand [13] that if Δb vanishes only at x = y = 0 and is a polyhomogeneous function of (x, y), that is, Δb(r^{k}x, r^{l}y) ≡ r^{n}Δb(x, y) for some positive integers k, l, n, then L is analytic hypoelliptic. In our example λ = Δb is polyhomogeneous for k = 4, but not for larger k.

Since [X, Y] = λ(x, y)∂_{t} and λ vanishes only on the line {(x, y, t) : x = y = 0} in ℝ^{3}, the bracket hypothesis is satisfied to order 2 everywhere except on that line. Where x = y = 0, it is satisfied to order precisely m = 6, because of the presence of the term x^{2}y^{2} in λ. The characteristic variety Σ ⊂ T^{∗}ℝ^{3} of L is a manifold of codimension two. It is symplectic, at every point at which the principal symbol of [X, Y] is nonzero. Thus Σ is symplectic everywhere except on the submanifold Σ′ ⊂ Σ given by Σ′ = {(x, y, t; ξ, η, τ) : x = y = ξ = η = 0}. Hence by virtue of the theorem of Tartakoff and of Treves, L is microlocally analytic hypoelliptic everywhere except possibly on Σ′.

Any particular iterated commutator of X, Y has a principal symbol which either vanishes identically on Σ′, or vanishes nowhere on Σ′. Therefore the stratification defined by Treves [22] takes the simple form Σ ⊃ Σ′ ⊃ ∅. Because Σ′ is itself a symplectic manifold, his conjecture then predicts that L should be analytic hypoelliptic.

3This lower order term is not insignificant, but the questions of interest regarding (X+iY)(X−iY)

concern its microlocal hypoellipticity properties in a subset of the cotangent bundle, where its properties coincide with those of X^{2} + Y^{2} in all known cases.

For simplicity we set k = 6 for the remainder of the discussion. All results generalize to k ≥ 6, but the details of the analysis change slightly.

We will establish two contrasting conclusions concerning this example. Set p_{T} = (0, 0, 0; 0, 0, T) and q_{T} = (0, 0, 1; 0, 0, T), where T ∈ ℝ^{+} is a large parameter.

Proposition 5.1. There exists c > 0 such that for each large T there exists a function φ_{T} which is microlocally Lipschitz relative to {X, Y }, and satisfies |φ(q_{T}) − φ(p_{T})| ≥ cT.

Proposition 5.2. It is not the case that there exist c > 0 and a one parameter family of functions φ_{T} such that for each large T ∈ ℝ^{+}, φ_{T} is microlocally Lipschitz relative to {X, Y }, |φ_{T} (q_{T}) − φ_{T} (p_{T})| ≥ cT, and moreover

|∂_{x,y,t}^{α}∂_{ξ,η,τ}^{β}φ_{T} | ≤ C_{α,β}(1 + |(ξ, η, τ)|)^{1−|β|} for all |α| + |β| ≤ 5.

Thus whether our philosophy predicts analytic hypoellipticity or not, depends on precisely which variant of ρ_{L} we consider. This author ventures no conjecture as to whether this particular operator is analytic hypoelliptic.

We first prove Proposition 5.1. We will construct a single function φ which has the required properties simultaneously for all T. Consider a function

φ(x, y, t; ξ, η, τ) = (t + h)τ + f · (ξ − τb_{y}) + g · (η + τb_{x})

where f, g, h are functions of x, y alone. Our aim is construct f, g, h so that φ is microlocally Lipschitz relative to {X, Y }. Since φ(q) − φ(p) = T − 0 = T, we can then conclude that ρ_{L}(p, q) ≥ cT.

Because the characteristic variety of L is {x = y = ξ = η = 0} and because the function (x, y, t; ξ, η, τ) ↦ τ is annihilated by both H_{X}, H_{Y}, it is easy to conclude using the same function φ that more generally, for any separated points p, q, one has ρ_{L}(p, q) ≥ cρ_{0}(p, q).

The Hamiltonian vector fields associated to X, Y are

H_{X} = ∂_{x} − b_{y}∂_{t} + τb_{yx}∂_{ξ} + τb_{yy}∂_{η},
H_{Y} = ∂_{y} + b_{x}∂_{t} − τb_{xx}∂_{ξ} − τb_{xy}∂_{η}.

Thus

H_{X}φ = −τb_{y} − τfb_{xy} + τgb_{xx} + τfb_{xy} + τgb_{yy} + τh_{x} plus error terms
= τ[−b_{y} + gΔb + h_{x}] plus error terms
H_{Y} φ = −τfb_{yy} + τgb_{xy} + τb_{x} − τb_{xx}f − τb_{xy}g + τh_{y}  plus error terms
= τ[b_{x} − fΔb + h_{y}] plus error terms.

Each of the aforementioned error terms is a product of one of f_{x}, f_{y}, g_{x}, g_{y} with either (ξ − τb_{y}) or (η + τb_{x}). The latter two quantities are the principal symbols of X, Y, hence are dominated by σ̃. If f, g are constructed so as to be Lipschitz continuous functions of (x, y), then each error term will be majorized by σ̃. Since our goal is to have |H_{X}φ| + |H_{Y} φ| ≤ Cσ̃, it therefore suffices to construct Lipschitz functions f, g and a function h such that

b_{y} − gΔb = h_{x} and − b_{x} + fΔb = h_{y}. (20)

Recalling that Δb = λ, a necessary condition for (20) is that b_{yy} − (gλ)_{y} = −b_{xx} + (fλ)_{x}, which may be rewritten as

λ = (fλ)_{x} + (gλ)_{y}. (21)

Conversely, if f, g are bounded solutions of (21) then there exists a solution h of (20) in the sense of distributions. But the left hand sides of both equations in (20) are bounded since b is a polynomial, so h is Lipschitz after possible redefinition on a set of measure zero; in fact h ∈ C^{1,1} when f, g are Lipschitz. Thus in order to show that φ is microlocally Lipschitz relative to {X, Y }, it suffices to prove the existence of Lipschitz continuous (in the ordinary sense) solutions f, g of (21).

Lemma 5.3. There exist Lipschitz continuous functions f, g, defined in a neighborhood of the origin in ℝ^{2}, satisfying λ ≡ (fλ)_{x} + (gλ)_{y}.

The conclusion is to be interpreted as equality almost everywhere. Note that if λ were homogeneous or polyhomogeneous, then there would exist a solution of the form f = k_{1}x, g = k_{2}y for certain constants k_{j}, by Euler’s identity. It is a relatively simple matter to construct Lipschitz solutions of λ = f · λ_{x} + g · λ_{y} in the present case, but that is not what is required.

Proof. We write A ≲ B to mean that the ratio A/B of nonnegative functions is bounded above by a finite constant, and A ∼ B to mean that A ≲ B and B ≲ A. Different constructions will be used in different regions of the plane. Consider first the region Γ_{1} where |x| ≤ 2|y|. We set g ≡ 0 and solve (fλ)_{x} = λ by defining

f(x, y) = λ(x, y)^{−1} ∫_{0}^{x} λ(s, y) ds.

In this region λ ∼ y^{6} + x^{2}y^{2}, so | ∫_{0}^{x} λ(x, y) ds| ≲ |x|λ(x, y). Hence |f(x, y)| ≲ |x|. In analyzing ∇f it will be useful to note that in Γ_{1},

|λ_{x}| ≲ λ/|x| and |λ_{y}| ≲ λ/|y|.

To analyze ∇f consider first

f_{x} = 1 − λ^{−2}λ_{x} · ∫_{0}^{x} λ(s, y) ds.

Since |λ_{x}| ≲ λ/|x| and the integral is ≲ |x|λ, the second term is uniformly bounded. Similarly

f_{y} = −λ^{−2}λ_{y} ∫_{0}^{x} λ + λ^{−1} ∫_{0}^{x} λ_{y}(s, y) ds.

In absolute value the first term is ≲ λ^{−2} · (λ/|y|) · (|x|λ), which is uniformly bounded in Γ_{1}. The absolute value of the second term is ≲ λ^{−1} ∫_{0}^{x} λ/|y| ds ≲ |x|/|y|, so likewise is bounded.

Fix an auxiliary function ϕ which is homogeneous of degree zero in ℝ^{2}, is C^{∞} except at the origin, is identically equal to one where |x| ≤ |y|/2, and is supported where |x| ≤ |y|. Set f̃ = f · ϕ. This formula makes sense only where |x| ≤ |y|, but we extend the definition by setting f̃ ≡ 0 where |x| > |y|. Then f̃ is Lipschitz, because |∇ϕ(z)| ≲ |z|^{−1} and |f(z)| ≲ |z|. We claim that where |y|/2 ≤ |x| ≤ |y| and |(x, y)| is sufficiently small,

|∇^{α}f̃(z)| ≤ C_{α}|z|^{1−|α|}

for every multi-index α. Because ϕ is homogeneous and smooth except at the origin, it suffices to verify this for f. One has explicitly

f(x, y) = \frac{xy^{6} + x^{3}y^{2}/3 + x^{7}/7}{y^{6} + x^{2}y^{2} + x^{6}}.

In the conic region in question, λ(z) ∼ |z|^{4}, the terms x^{6}, y^{6} in the denominator being comparatively negligible. Hence |∇^{α}f(z)| ≲ |z|^{1−|α|} where |x| ∼ |y|. The claim follows.

In the (overlapping) region where |y| ≤ 2|x|, we solve instead (gλ)_{y} = λ, taking f ≡ 0. Because λ is a symmetric function of (x, y), conclusions parallel to those above may be obtained. Define g̃(x, y) = g(x, y)ϕ(y, x), and again define g̃ ≡ 0 where |y| > |x|. We have then λ ≡ (f̃λ)_{x} + (g̃λ)_{y} where |x| ≤ |y|/2 and also where |y| ≤ |x|/2.

Define

λ̃ = λ − (f̃λ)_{x} − (g̃λ)_{y}.

λ̃ is supported in the conic region Γ_{3} where |y|/2 ≤ |x| ≤ 2|y|. It satisfies |∇^{α}λ̃(z)| ≲ |z|^{4−|α|} for all z ∈ Γ_{3} and all α.

In order to complete the proof of the lemma, it suffices to construct Lipschitz functions F, G satisfying

λ̃ = (F λ)_{x} + (Gλ)_{y}.

To accomplish this we set F = xh and G = yh and solve λ̃ = (xhλ)_{x} + (yhλ)_{y} for the single unknown h. In polar coordinates (r, θ) the equation becomes

λ̃ = λrh_{r} + h · (2λ + xλ_{x} + yλ_{y}).

Define

β(x, y) = (2 + xλ_{x}/λ + yλ_{y}/λ)

and note that 2 ≤ β and moreover that in Γ_{3}, |∇^{α}β(z)| ≲ |z|^{−|α|}. The equation to be solved is

h_{r} + hβ/r = r^{−1}λ̃/λ. (22)

It suffices to produce a bounded solution whose gradient is ≲ r^{−1}.

Note for future use that

|β_{θ}(r, θ)| ≲ r

in Γ_{3}, not merely ≲ 1. The fact that λ is homogeneous of degree 4 modulo higher order terms and nonvanishing in Γ_{3} leads by Euler’s identity to the conclusions β(r, θ) = 6 + O(r) and ∇β = O(1) in that cone. This strengthened bound is essential to our construction.

Define next

b(r, θ) = ∫_{1}^{r} β(ρ, θ), dρ/ρ.

Then b(r, θ) < 0 for r < 1, and |b(r, θ)| ∼ log r^{−1}. Because β/ρ ≥ 2/ρ, e^{b}(ρ, θ) ≲ ρ^{2} as ρ → 0.

Define a solution h of (22) by

h(r, θ) = e^{−b} ∫_{0}^{r} e^{b}(ρ, θ)\frac{λ̃}{λ}(ρ, θ) \frac{dρ}{ρ};

the integral converges since λ̃/λ is bounded and e^{b} ≲ ρ^{2}. Formally h satisfies the required equation. Note that h is supported in Γ_{3}, since the integrand vanishes identically outside it.

To complete the proof of the lemma it suffices to show that |h| ≲ 1, and that |∇h(r, θ)| ≲ r^{−1}. Because we have seen above that λ̃/λ is bounded, an upper bound for |h| is e^{−b} ∫_{0}^{r} e^{b} dρ/ρ. Since b_{r} is between 2/r and C/r, the factor e^{b} is monotone increasing, and e^{b}(r/2, θ) ≤ \frac{1}{4}e^{b}(r, θ). Consequently this last integral has the same order of magnitude as e^{b}, and hence h is uniformly bounded.

From the boundedness of h and the differential equation (22) it follows immediately that the partial derivative h_{r} is ≲ r^{−1} in absolute value. It remains to show that |h_{θ}| ≲ 1. Differentiation in the definition of h yields several terms; the simplest is −b_{θ} times h. We have

|b_{θ}| ≤ ∫_{r}^{1} |β_{θ}| dρ/ρ,

and |β_{θ}(ρ, θ)| ≤ ρ|∇β| ≲ ρ in Γ_{3}, as observed above. Thus b_{θ} is uniformly bounded. Hence this simplest term is ≲ h, hence uniformly bounded.

A second term arises when on differentiating h with respect to θ, the derivative falls upon the factor of e^{b} inside the integral. An additional factor of b_{θ} results, causing no harm since it is uniformly bounded; the analysis indicated above for h itself applies also to this term. The third and last term arises when the derivative falls on the factor of λ̃/λ inside the integral. Now

|∂_{θ}(λ̃/λ)(r, θ)| ≤ Cr|∇(λ̃/λ)| ≲ 1

in Γ_{3}, since |∇λ̃| ≲ |z|^{4−|α|} in Γ_{3}, the same holds for λ, and |z|^{4} ≲ λ(z) there. Thus the same analysis applies once more. Hence |∇h| ≲ r^{−1}, and the proofs of both the lemma and Proposition 5.1 are complete.

Proof of Proposition 5.2. Suppose to the contrary that functions φ_{T} possessing the indicated properties were to exist. Note that H_{X}, H_{Y} are homogeneous of degree zero with respect to the dilations (x, y, t; ξ, η, τ) ↦ (x, y, t; rξ, rη, rτ) for r ∈ ℝ^{+}. By applying Arzela-Ascoli and a rescaling argument based on these dilations and homogeneity to the collection {φ_{T} }, we could then extract a single function φ which in any fixed relatively compact subset of T^{∗}ℝ^{3} minus the zero section belonged to C^{5} and satisfied φ(p) ≠ φ(q), where p = p_{1}, q = q_{1}, and

|H_{X}φ| + |H_{Y} φ| ≤ |ξ − τb_{y}| + |η + τb_{x}|. (23)

The vector fields H_{X}, H_{Y} are tangent to each level set of τ, so the restriction of φ to {τ = 1} must satisfy the preceding inequality. Henceforth we restrict attention to that level set, freezing τ = 1. Since φ(p) ≠ φ(q), there must exist t such that ∂_{t}φ(0, 0, t; 0, 0, 1) ≠ 0. Both H_{X}, H_{Y} are translation invariant with respect to t, so we may translate the coordinates so that ∂_{t}φ(0, 0, 0; 0, 0, 1) ≠ 0.

Next, expanding in Taylor series with respect to t, ξ, η about the origin and rewriting the result slightly gives

φ(x, y, t; ξ, η, 1) = h + γt + f · [ξ − b_{y}] + g · [η + b_{x}] + E

where E(x, y, t; ξ, η) = O(t, ξ, η)^{2} and the coefficients γ, f, g, h are functions of x, y alone. Moreover γ(0, 0) ≠ 0 because ∂_{t}φ ≠ 0; by dividing through by a constant we may assume that γ(0, 0) = 1.

Applying H_{X}, H_{Y}, evaluating at t = ξ = η = 0, and invoking (23) yields now the equations (20) that we arrived at in the proof of Proposition 5.1. Consequently λ ≡ (fλ)_{x} + (gλ)_{y}.

Consider the Taylor expansions of both sides of this last equation about the origin, and compare terms of equal degrees. By examining terms homogeneous of degree three, one finds immediately that f(0) = g(0) = 0. Equality for degree 4 forces f = c_{1}x + O^{2}(x, y) and g = c_{2}y + O^{2}(x, y), where 3c_{1} + 3c_{2} = 1, in order to reproduce the term x^{2}y^{2} on the left without introducing other monomials of degree 4. The contribution to (fλ)_{x} + (gλ_{y}) of the sum of all terms homogeneous of degree 2 in the Taylor expansions of f, g is a homogeneous polynomial of degree 5 plus a remainder which is O^{7}(x, y). The part of degree 5 must vanish identically, since no such terms are present in the expansion of λ. Consider finally the coefficient of x^{6} on the right. It equals 7c_{1} + c_{2}, for the monomials x^{2}y^{2} and y^{6} in λ cannot possibly lead to any x^{6} term on the right. Similarly the coefficient of y^{6} equals c_{1} + 7c_{2}. Therefore 7c_{1} +c_{2} = 1 = c_{1} +7c_{2}. The unique solution is c_{1} = c_{2} = 1/8, but this is incompatible with 3c_{1} + 3c_{2} = 1. This completes the proof of nonexistence of φ.

Summary. Although a conjecture linking the growth of ρ_{L} to hypoellipticity, in various function spaces, is consistent with a wide variety of examples, Example 9 suggests that this conjecture may be overly simplistic. Another reason for wariness is that such a conjecture takes no account of the behavior of the symbols σ_{X_{j}} in the complexified phase space. To decide whether Example 9 is analytic hypoelliptic might be quite illuminating.

Acknowledgement. The author is indebted to Chun Li for a suggestion used in the proof of Lemma 5.3, and to Charles Fefferman for kindly explaining [11] and [18].

References

[1] M. S. Baouendi and C. Goulaouic, Nonanalytic-hypoellipticity for some degenerate elliptic operators, Bulletin AMS 78 (1972), 483-486.

[2] A. Bove and D. Tartakoff, Optimal non-isotropic Gevrey exponents for sums of squares of vector fields, preprint.

[3] M. Christ, The Szegö projection need not preserve global analyticity, Annals of Math. 143 (1996), 301-330.

[4] , Intermediate optimal Gevrey exponents occur, Comm. Partial Differential Equations 22 (1997), 359-379.

[5] , Analytic hypoellipticity in dimension two, MSRI preprint 1996-009, submitted for publication.

[6] , Examples pertaining to a conjecture on Gevrey class hypoellipticity, Math. Research Letters, to appear.

[7] , Hypoellipticity in the infinitely degenerate regime, preprint October 1997, submitted for publication.

[8] M. Derridj and C. Zuily, Sur la régularité Gevrey des opérateurs de Hörmander, J. Math. Pures et Appl. 52 (1973), 309-336.

[9] V. S. Fediǐ, On a criterion for hypoellipticity, Math. USSR Sb. 14 (1971), 15-45.

[10] C. Fefferman, The uncertainty principle, Bulletin AMS 9 (1983), 129–206.

[11] , Symplectic subunit balls and algebraic functions, proceedings of symposium in honor of A. P. Calderón, to appear.

[12] C. Fefferman and D. H. Phong, Subelliptic eigenvalue problems, in Conference on Harmonic Analysis in Honor of Antoni Zygmund, Vol I,II, Chicago 1981, Wadsworth Math. Ser., Wadsworth, Belmont, Ca. 1983, 590-606.

[13] A. Grigis and J. Sjöstrand, Front d’onde analytique et sommes de carrés de champs de vecteurs, Duke Math. J. 52 (1985), 35-51.

[14] V. V. Grušin, A certain class of elliptic pseudodifferential operators that are degenerate on a submanifold, Mat. Sbornik 84 (1971), 163-195, = Math. USSR Sbornik 13 (1971), 155-185.

[15] T. Matsuzawa, Gevrey hypoellipticity for Grushin operators, Pub. RIMS Kyoto University, to appear.

[16] G. Métivier, Non-hypoellipticité analytique pour D_{x}^{2} + (x^{2} + y^{2})D_{y}^{2}, Comptes Rendus Acad. Sci. Paris 292 (1981), 401-404.

[17] A. Nagel, E. M. Stein and S. Wainger, Balls and metrics defined by vector fields I: Basic properties, Acta Math. 155 (1985), 103-147.

[18] A. Parmeggiani, Subunit balls for symbols of pseudodifferential operators, Advances in Math., to appear.

[19] A. Sánchez-Calle, Fundamental solutions and geometry of the sum of squares of vector fields, Invent. Math. 78 (1984), 143-160.

[20] F. Treves, Analytic hypo-ellipticity of a class of pseudodifferential operators with double characteristics and applications to the ¯∂–Neumann problem, Comm. Partial Differential Equations 3 (1978), 475-642.

[21] , Introduction to Pseudodifferential and Fourier Integral Operators, Volume 1, Plenum Press, New York, 1980.

[22] , Symplectic geometry and analytic hypo-ellipticity, preprint.

Michael Christ, Department of Mathematics, University of California, Berkeley 94720, Berkeley, CA USA

  E-mail address: mchrist@math.berkeley.edu