Functional Analysis – Introduction. Part II

In a previous post we discussed some of the history of functional analysis and we also said some vague things about its role in mathematics. In this second part of the introduction we will see an example of the spirit of functional analysis in action, by taking a close look at the Stone-Weierstrass approximation theorem.

1. The Weierstrass approximation theorem

In 1885 Karl Weierstrass proved the following theorem:

Weierstrass Theorem: Let $latex f : [a,b] rightarrow mathbb{R}$ be a continuous function. Then for every $latex epsilon > 0$ there exists a polynomial $latex p$ (with real coefficients) such that for all $latex x in [a,b]$,

$latex |p(x) – f(x)|< epsilon .$

The theorem can be phrased differently, by saying that the polynomials are dense in the space of continuous functions with respect to the topology of uniform convergence. I think that this is a remarkable theorem. You see, only thirteen years before Weierstrass proved this theorem he broke the news that continuous functions can be much crazier then anyone ever thought, by constructing his famous Weierstrass function – a continuous function that is nowhere differentiable. But now, in 1885, Weierstrass actually showed that for many purposes, in practice (well, at least theoretically), continuous functions could be taken to be polynomials. We will see the usefulness of this theorem, directly and indirectly, at least at two points in the lectures.

2. The Stone-Weierstrass theorem – real version

Given a topological space $latex X$, the question whether or not a family of functions can approximate uniformly to any given precision any continuous function on $latex X$ is interesting and important. In a few lectures, when we will study Fourier series, we will need to use the following theorem:

Theorem (trigonometric approximation theorem): Let $latex f: mathbb{R}^d rightarrow mathbb{R}$ be a continuous, $latex mathbb{Z}^d$-periodic function. Then for every $latex epsilon > 0$ there exists a trigonometric polynomial $latex q$ such that for all $latex x in mathbb{R}^d$,

$latex |q(x) – f(x)|< epsilon .$

Two explanations are in order. A  $latex mathbb{Z}^d$-periodic function is a function $latex f$ such that $latex f(x+n) = f(x)$ for all $latex x in mathbb{R}^d$ and $latex n in mathbb{Z}^d$. A trigonometric polynomial is a function $latex q$ of the form

$latex q(x) = sum_{n} a_n cos(2 pi ncdot x) + b_n sin(2 pi n cdot x) .$

(a finite sum). As it happens many times in mathematics, it turns out that the nicest way to obtain  this theorem is to a consider a vastly more general problem. The solution of this more general problem is the Stone-Weierstrass Theorem, which we shall presently state and prove. Before doing so let us describe the setting for this theorem.

Let $latex X$ be a compact Hausdorff space. We will denote by $latex C_mathbb{R}(X)$ the space of continuous, real-valued functions on $latex X$, and by $latex C(X)$ the space of continuous, complex-valued functions on $latex X$. On both of these space we define the sup-norm of a function $latex f$ to be

$latex |f|_infty = sup_{x in X}|f(x)| .$

The quantity $latex |f-g|_infty$ is considered to be distance between the two functions $latex f$ and $latex g$. This distance makes both $latex C_mathbb{R}(X)$ and $latex C(X)$ into complete metric spaces. Both these spaces are vector spaces over the appropriate field, in fact they are algebras, with the usual operations of pointwise addition and multiplication. If $latex A$ is a subspace of $latex C_mathbb{R}(X)$ or of $latex C(X)$ then it is said to be a subalgebra if for all $latex f,g in A$, $latex fg$ is also in $latex A$. A subalgebra $latex A$ is said to separate points if for all $latex x,y in X$ there exists some $latex f in A$ such that $latex f(x) neq f(y)$. A subalgebra is said to be closed if it is a closed subset with respect to the metric.

Stone-Weierstrass Theorem (real version): Let $latex A$ be a closed subalgebra of $latex C_mathbb{R}(X)$ which contains the constant functions and separates points. Then $latex A = C_mathbb{R}(X)$.

To obtain the theorem above regarding approximation using trigonometric polynomials, you first note that, due to standard trigonometric identities, the trigonometric polynmials form an algebra. Next, you think about and realize that the $latex mathbb{Z}^d$-periodic functions on $latex mathbb{R}^d$ can be identified with the continuous functions on the $latex d$-torus $latex mathbb{T}^d = mathbb{R}^d / mathbb{Z}^d$. You then put $latex X = mathbb{T}^d$, and take $latex A$ to be the norm closure of the trigonometric polynomials in $latex C_mathbb{R}(X)$, noting that the norm closure of an algebra is an algebra. To apply the Stone-Weierstrass theorem it suffices now to show that the trigonometric polynomials separate points, and this is elementary. That concludes the proof of the theorem on approximation by trigonometric polynomials.

3. Proof of the Stone-Weierstrass Theorem

Let us isolate three lemmas before we reach the main argument of the proof.

Lemma 1: For every pair of distinct points $latex x,y in X$, and every $latex a,b in mathbb{R}$, there exists a function $latex g in A$ such that $latex g(x) = a$ and $latex g(y) = b$.

Proof: Exercise.

Lemma 2: If $latex f in A$ then the function $latex |f|$ is also in $latex A$.

Here $latex |f|$ denotes the function that sends every $latex x in X$ to $latex |f(x)|$.

Proof: Let $latex epsilon > 0$ be given. We will find a function $latex g in A$ such that $latex |g- |f||_infty < epsilon$. Since $latex A$ is closed and since $latex epsilon$ is arbitrary, this will show that $latex |f| in A$.

Let $latex I = [-|f|_infty, |f|]$. By the Weierstrass Theorem there exists a polynomials $latex p$ such that $latex sup_{t in I} |p(t) – |t|| < epsilon$. Put $latex g = p circ f$. Since $latex A$ is an algebra and $latex p$ is a polynomial, $latex g in A$. Thus

$latex |g-|f||_infty = sup_{x in X} |p(f(x)) – |f(x)|| leq sup_{t in I} |p(t) – |t|| < epsilon$

as required. 

Lemma 3: If $latex f,g in A$, then the functions $latex f wedge g$ and $latex f vee g$ are also in $latex A$.

Here  $latex f wedge g$ denotes the function that sends every $latex x in X$ to $latex min{f(x),g(x)}$, and $latex f vee g$ denotes the function that sends every $latex x in X$ to $latex max{f(x),g(x)}$.

Proof: This follows immediately from Lemma 2 together with the formulas $latex a wedge b = frac{a+b-|a-b|}{2}$ and $latex a vee b = frac{a+b+|a-b|}{2}$, which hold true for all real $latex a$ and $latex b$.

Completion of the proof of the Stone-Weierstrass Theorem. Let $latex f in C_mathbb{R}(X)$. We must show that $latex f in A$. It suffices, for every $latex epsilon >0$,  to find $latex h in A$ such that $latex |f-h|_infty < epsilon$.

We start by constructing, for every $latex x,y in X$, a function $latex f_{x,y} in A$ such that $latex f_{xy}(x) = f(x)$ and $latex f_{xy}(y) = f(y)$. This is possible thanks to Lemma 1.

Next we produce, for every $latex x in X$, a function $latex g_x in A$ such that $latex g_x(x) = f(x)$ and $latex g_x(y) < f(y) + epsilon$ for all $latex y in X$. This is done as follows. For every $latex y in X$, let $latex U_y$ be a neighborhood of $latex y$ in which $latex f_{xy} < f + epsilon$. The compactness of $latex X$ ensures that there are finitely many of these neighborhoods, say $latex U_{y_1}, ldots, U_{y_m}$, that cover $latex X$. Then $latex g_x = f_{xy_1} wedge ldots wedge f_{xy_m}$ (which is in $latex A$, thanks to Lemma 3) does the job.

Finally, we find $latex h in A$ such that $latex |h(x)-f(x)| < epsilon$ for all $latex x in X$. For every $latex x in X$ let $latex V_x$ be a neighborhood of $latex x$ where $latex g_x > f – epsilon$. Again we find a finite cover $latex V_{x_1}, ldots, V_{x_n}$ and then define $latex h = g_{x_1} vee ldots vee g_{x_n}$. This function lies between $latex f + epsilon$ and $latex f – epsilon$, so it satisfies $latex |h(x)-f(x)| < epsilon$ for all $latex x in X$, and the proof is complete.

Remark: In the proof, we never used the assumption that $latex X$ is Hausdorff. But from the way the theorem is stated it seems like it only expects to be invoked in the compact Hausdorff case. Can you figure out why the theorem is stated with this assumption?

4. The Stone-Weierstrass theorem – complex version

Often, one finds it more convenient to study or to use the algebra $latex C(X)$ of complex valued functions. It turns out that it is much harder for a sub-algebra of functions to be dense in $latex C(X)$. Consider for example $latex A(mathbb{D})$ which is defined to be the closure of polynomials in $latex C(overline{mathbb{D}})$ (here $latex mathbb{D}$ denotes the open unit disc in $latex mathbb{C}$, and $latex overline{mathbb{D}}$ denotes its closure). Certainly, $latex A(mathbb{D})$ is a complex algebra which contains the constants and separates points. However, every element of $latex A(mathbb{D})$ is holomorphic in $latex mathbb{D}$, so this algebra is quite far from being the entire algebra $latex C(overline{mathbb{D}})$. To make the Stone-Weierstrass Theorem work in the complex valued case one needs to add another assumption, which is very simple, but turns out to be deeper than it seems at first.

Definition: A subspace $latex S subseteq C(X)$ is said to be self-adjoint if for every $latex f in S$, the complex conjugate $latex overline{f}$ is also in $latex S$.

Stone-Weierstrass Theorem (complex version): Let $latex A$ be a closed and self-adjoint subalgebra of $latex C(X)$ which contains the constant functions and separates points. Then $latex A = C(X)$.

Proof: From the assumptions that $latex A$ is a subspace that separates points it follows that $latex textrm{Re}A$, the space of all functions of the form $latex textrm{Re}f$, $latex f in A$, is a real subspace that separates points. From the self-adjointness assumption it follows that $latex textrm{Re}A subseteq A$, and from closedness of $latex A$ is follows now that $latex textrm{Re}A$ is closed. Thus $latex textrm{Re}A$ is a closed real algebra that contains the constants and separates points. By the (real) Stone-Weierstrass Theorem, $latex textrm{Re}A = C_mathbb{R}(X)$. It follows that every real valued continuous function on $latex X$ is in $latex A$. Symmetrically, every imaginary valued continuous function on $latex X$ is in $latex A$, so $latex C(X) = A$.

5. Concluding remarks

The Stone-Weierstrass Theorem and the way that we have applied it up serve as a baby example of functional analysis at work. We had a concrete approximation problem, which was solved, as it happens often in mathematics, by considering a vastly more general approximation problem. Considering a more general problem serves two purposes. First, after we have proved the result we have a ready-to-use tool that will be applicable in many situations. Second, by generalizing the problem we strip it away of the irrelevant details (for examples the particular nature of the functions we are trying to approximate with or the particular nature of the space on which they live) and we are left with the essence of the problem. Sometimes this leads to a very elegant proof (like above), and sometimes not.

To prove the theorem it was convenient to employ the language of functional analysis, namely, to introduce a norm and to consider the problem inside an algebra which is also a metric space. It was convenient to consider a closed algebra $latex A$, even though there was no closed algebra in the original problem (the trigonometric approximation theorem), and even though this closed algebra turned out to be the entire space of continuous functions. The student might wish to review the proof to see how this streamlined the argument.

Note that we did not obtain a new and “abstract” proof of the Weierstrass Theorem – in fact, the Weierstrass Theorem was used in the proof of the Stone-Weierstrass Theorem. This is not at all surprising, as it is an instance of the following guiding principle that I have come to believe in (please take this with a grain of salt):

There is no analysis without hard analysis.

What I mean by this is that you will never get an interesting theorem of substance, which applies to concrete cases in analysis, that does not involve some kind of hard analysis, some epsilonaustics, some sweat. In our example, the hard analysis comes in the proof of the Weierstrass Theorem. The Stone-Weierstrass Theorem, which has a very soft proof, then spares us the use of hard techniques when it gives us the trigonometric approximation theorem for free. But the hard analysis cannot be eliminated entirely.

I know of a different proof of the Stone-Weierstrass Theorem that does not use the Weierstrass Theorem, which relies on the Hahn-Banach and Krein-Milman theorems, but that proof uses the structure of the dual of $latex C(X)$, and that I consider another bit of hard analysis.

When we moved from the setting of the trigonometric approximation theorem to the setting of the Stone-Weierstrass Theorem we abstracted away the topological space on which the functions live, and we only cared that we have continuous functions on some compact space. In the next lecture, and in most of the rest of the course, we will abstract away everything, and we will consider abstract spaces satisfying certain axioms, and we will not care usually what the points of these spaces are.

We will begin with Hilbert spaces.