(Quick announcement: all lectures will from now on take place in room 201).
In the previous lecture, we learned the very basics of Hilbert space theory. In this lecture we shall go one little bit further, and prove the basic structure theorems for Hilbert spaces.
0. Continuity of the inner product
Exercise A: Let $latex G$ be an inner product space. Prove that the inner product is continuous with respect to the norm: if $latex x_n rightarrow x$ and $latex y_n rightarrow y$ in $latex G$, then $latex (x_n,y_n) rightarrow (x,y)$. Conclude in particular that if $latex x_n rightarrow x$ then $latex |x_n| rightarrow |x|$.
1. Orthogonality
Definition 1: Let $latex G$ be an inner product space.
(a) Two vectors $latex x,y in G$ are said to be orthogonal, denoted $latex x perp y$, if $latex (x,y) = 0$.
(b) A set of non-zero vectors $latex {e_i}_{i in I} subseteq G$ is said to be an orthogonal set if $latex e_i perp e_j$ for all $latex i neq j$.
(c) An orthogonal set $latex {e_i}_{i in I} subseteq G$ is said to be an orthonormal set if $latex |e_i| = 1$ for all $latex i neq j$. An orthonormal set is sometimes called and orthonormal system.
The following two easy propositions show how the geometry of inner product spaces has some close similarities with Euclidean geometry. These similarities invite mathematicians to use their geometric intuition when working in inner product spaces, and make these spaces especially lovable.
Proposition 2 (Pythagorean identity): In an inner product space, the following hold
(a) If $latex x perp y$ then $latex |x + y|^2 = |x|^2 +|y|^2$.
(b) If $latex {e_i}_{i =1}^n$ is a finite orthogonal set then
$latex |sum_i e_i |^2 = sum_i |e_i|^2 .$
Proof: (a) is a special case of (b), which in turns follows from
$latex |sum_i e_i|^2 = (sum_i e_i, sum_j e_j) = sum_{i,j=1}^n (e_i, e_j) = sum_i |e_i|^2$.
Example: Let $latex G = C[0, 1]$ with the usual inner product. The set $latex {e^{2pi inx}}_{n=-infty}^infty$ is an orthonormal set:
$latex (e^{2pi imx}, e^{2pi inx}) = int_0^1 e^{2pi i(m-n)x} dx $
and this is equal to $latex 1$ if $latex m=n$ and to $latex frac{1}{2pi i(m-n)} e^{2pi i(m-n)x}Big|_0^1 = 0$ if $latex m neq n$. The set $latex {1, sin 2pi n x, cos 2pi nx}_{n=1}^infty$ is an orthogonal set, but not orthonormal. These two systems are also orthogonal sets in the larger space $latex H = L^2[0,1]$.
The Pythagorean identity is a very special property of inner product spaces, and it gets used all the time. Another immediate identity that holds in inner product spaces is the following.
Proposition 3 (parallelogram law): For any $latex x,y$ in an inner product space, the following holds:
$latex |x+y|^2 + |x-y|^2 = 2 |x|^2 + 2|y|^2 .$
This identity follows only from the fact that the norm in an inner product space is defined by a sesquilinear form. The parallelogram law differs from the Pythagorean identity in that it is stated only in terms of the norm, and makes no mention of the inner product. Thus, it can be used to show that there are norms not induced by an inner product. Though it does not get used so much, we will soon use the parallelogram law to prove a some of the most fundamental theorems in Hilbert space theory (see Section 2).
In the study of of vector spaces of finite dimension, bases play an important role. Recall that $latex v_1, ldots, v_n$ is a basis for a vector space $latex G$ if for all $latex x in G$ there exist unique scalars $latex c_1, ldots, c_n$ such that
$latex x = sum_{i=1}^n c_i v_i .$
When dealing with finite dimensional inner product spaces, orthonormal bases are particularly handy, because if $latex v_1, ldots, v_n$ is an orthonormal basis then the above $latex c_i$ is given by $latex (x,v_i)$. Furthermore one has $latex |x|^2 = sum_{i=1}^n |(x,v_i)|^2$, and if $latex y = sum_i d_i v_i$ then
$latex (x,y) = sum_{i=1}^n c_i overline{d}_i .$
The beautiful and useful fact is that all of this remains true in any Hilbert space. To explain what all of this means in infinite dimensional spaces requires a little care.
Definition 4: Let $latex I$ be any set, and let $latex {a_i}_{iin I}$ be a set of complex numbers. We say that the series $latex sum_{i in I} a_i$ converges to $latex s in mathbb{C}$, and we write $latex sum_{i in I} a_i = s$, if for all $latex epsilon >0$, there exists a finite set $latex F_0$ such that for every finite set $latex F subseteq I$ for which $latex F supseteq F_0$ ,
$latex |sum_{i in F}a_i -s|< epsilon .$
Definition 4*: Let $latex I$ be any set, and let $latex {x_i}_{iin I}$ be a set of elements in an inner product space $latex G$. We say that the series $latex sum_{i in I} x_i$ converges to $latex x in G$, and we write $latex sum_{i in I}x_i = x$, if for all $latex epsilon >0$, there exists a finite set $latex F_0$ such that for every finite set $latex F subseteq I$ for which $latex F supseteq F_0$ ,
$latex |sum_{i in F}x_i -x|< epsilon .$
Exercise B: Prove that a series $latex sum_{i in I} x_i$ converges to $latex x$ if and only if there exists a countable set $latex J = {j_1, j_2, ldots, }$ contained in $latex I$ such that (a) $latex x_i = 0$ if $latex i in I setminus J$; and (b) for any rearrangement $latex J’ = {j’_1, j’_2, ldots, }$ of $latex J$, the limit $latex lim_{N rightarrow infty} sum_{n=1}^N x_{j’_n} = x$ exists in $latex G$.
Exercise C: Suppose that $latex a_i geq 0$ for all $latex i$. Prove that $latex sum_{i in I} a_i$ converges and is bounded by $latex M$ if and only if the set of all finite sums $latex {sum_{i in F} a_i : F subseteq I textrm{ finite} }$ is bounded by $latex M$.
Proposition 5 (Bessel’s inequality): Let $latex {e_i}_{i in I}$ be an orthonormal set in an inner product space $latex G$, and let $latex x in G$. Then
$latex sum_{i in I} |(x,e_i)|^2 leq |x|^2 . $
Proof: Let $latex F subseteq I$ be finite. A computation shows that $latex sum_{i in F} (x,e_i) e_i$ is orthogonal to $latex x – sum_{iin F} (x,e_i) e_i$. But $latex x = sum_{iin F} (x,e_i)e_i + (x – sum_{iin F} (x,e_i)e_i)$, therefore, by the Pythagorean identity,
$latex |x|^2 = |sum_{iin F} (x,e_i)e_i|^2 + |x – sum_{iin F} (x,e_i)e_i|^2 ,$
thus (by Pythagoras once more) $latex sum_{iin F} |(x,e_i)|^2 leq |x|^2$. This holds for all $latex F$, so (invoking Exercise C) the assertion follows.
Exercise D: Deduce the Cauchy-Schwarz inequality from Bessel’s inequality. Did this involve circular reasoning?
2. Orthogonal decomposition and orthogonal projection
Definition 6: A subset $latex S$ in a vector space $latex V$ is said to be convex if for all $latex x, y in S$ and all $latex t in [0,1]$, the vector $latex tx + (1-t) y$ is also in $latex S$.
Lemma 7: Let $latex S$ be a closed convex set in a Hilbert space $latex H$. Then there is a unique $latex x in S$ of minimal norm.
Proof: Put $latex d:= inf{|y| : y in S}$. Let $latex {x_n}$ be a sequence in $latex S$ such that $latex |x_n| rightarrow d$. Applying the parallelogram law to $latex x_m/2$ and $latex x_n/2$, we find
$latex |frac{1}{2}(x_m – x_n)|^2 = 2|frac{1}{2}x_n|^2+2|frac{1}{2}x_m|^2 – |frac{1}{2}(x_n + x_m)|^2$
Now, $latex |frac{1}{2}(x_n + x_m)|^2 geq d^2$, thus, letting $latex m,n rightarrow infty$, we find that the right hand side must tend to zero, hence $latex {x_n}$ is a Cauchy sequence. Since $latex H$ is complete and $latex S$ closed, there is an $latex x in S$ such that $latex x_n rightarrow x$. By Exercise A, $latex |x| = d$, this proves the existence of a norm minimizer. To prove the uniqueness, assume that $latex y in S, |y| = d$. If we form the sequence $latex x, y, x, y, ldots$ then we have just seen above that this is a Cauchy sequence. It follows that $latex x = y$.
Theorem 8: Let $latex S$ be a closed convex set in a Hilbert space $latex H$, and let $latex x in H$. Then there exists a unique $latex y in S$ such that
$latex |x-y| leq |x – w|$
for all $latex w in S$.
Proof: Apply Lemma 7 to the convex set $latex S – x$. Any element in $latex S – x$ of minimal norm corresponds to a $latex y in S$ such that $latex |y-x|$ is minimal.
We call the element $latex y in S$ the best approximation for $latex x$ within $latex S$. We denote by $latex P_S$ the function that assigns to each $latex x in H$ the best approximation for $latex x$ within $latex S$. Since every subspace is convex, we immediately obtain the following.
Corollary 9: Let $latex M$ be a closed subspace of $latex H$, and let $latex x in H$. Then there exists a unique $latex y in M$ such that
$latex |x-y| leq |x – w|$
for all $latex w in M$.
Theorem 10: Let $latex S$ be a convex set in a Hilbert space $latex H$, and let $latex x in H$ and $latex y in S$. The following are equivalent.
- $latex y = P_S (x)$ (in other words, $latex y$ is the best approximation for $latex x$ within $latex S$).
- $latex Re(x – y, s-y) leq 0$ for all $latex s in S$.
Proof: Let $latex y = P_s(x)$. Then expanding the inequality
$latex |x – (ts + (1-t)y)|^2 geq |x – y|^2 ,$
(which holds for all $latex t in (0,1)$), we get
$latex |x-y|^2 – 2Re(x-y,t(s-y)) + |t(s-y)|^2 geq |x-y|^2 .$
Dividing by $latex t$ and cancelling some terms, we obtain $latex 2 Re (x-y, s-y) leq t |s-y|^2$. Thus 1 implies 2.
To get the implication 2 $latex Rightarrow$ 1, we write
$latex |x-s|^2 = |(x-y) – (s-y)|^2 = |x-y|^2 – 2 Re(x-y,s-y) + |s-y|^2 geq |x-y|^2 .$
Corollary 11: Let $latex M$ be a closed subspace in a Hilbert space $latex H$,
and let $latex x in H$ and $latex y in M$. The following are equivalent.
- $latex y = P_M (x)$ (in other words, $latex y$ is the best approximation for $latex x$ within $latex M$).
- $latex x – y perp m$ for all $latex m in M$.
Definition 12: Let $latex G$ be an inner product space, and let $latex S subseteq G$. We define $latex S^perp := {g in G : forall s in S . (s,g) = 0}$.
Exercise E: Prove that $latex S^perp$ is always a closed subspace, and that $latex S cap S^perp subseteq {0}$.
Theorem 12: Let $latex M$ be a closed subspace in a Hilbert space $latex H$. Then for every $latex x in H$ there is a unique $latex y in M$ and a unique $latex z in M^perp$ such that $latex x = y + z$.
Remark: The conclusion of the theorem is usually denoted shortly by $latex H = M oplus M^perp$.
Proof: Write $latex x = P_M x + (x – P_M x)$. Then $latex y:= P_M x in M$ by definition. Moreover, letting $latex z:= x – P_M x$, we have $latex (z,m) = 0$ by Corollary 11. Thus $latex z in M^perp$. For uniqueness, assume that $latex x = y + z = y’ + z’$. Then we have
$latex M ni y – y’ = z – z’ in M^perp ,$
and since $latex M cap M^perp = {0}$ we have $latex y = y’$ and $latex z = z’$.
Theorem 13: Let $latex S$ be a closed convex subset in a Hilbert space $latex H$. The map $latex P_S$ satisfies $latex P_S circ P_S = P_S$ , and $latex P_S (x) = x$ if and only if $latex x in S$. The map $latex P_S$ is linear if and only if $latex S$ is a subspace.
Exercise F: Prove Theorem 13.
A mapping $latex T$ satisfying $latex T circ T = T$ is called a projection. In the case that $latex S$ is a subspace, then $latex P_S$ is called the orthogonal projection onto $latex S$.
Theorem 14: Let $latex M$ be a subspace of a Hilbert space $latex H$. Then $latex M$ is closed if and only if $latex M = M^{perp perp}$.
Proof: Trivially, $latex M subseteq M^{perpperp}$ (“anything in $latex M$ is orthogonal to anything that is orthogonal to everything in $latex M$”) and we already noted that $latex M^{perpperp}$ is closed (Exercise E). So assume that $latex x in M^{perpperp}$, and write the orthogonal decomposition of $latex x$ with respect to the decomposition $latex H = M oplus M^perp$, that is $latex x = y + z$, with $latex y in M, z in M^{perp}$. But $latex y in M^{perpperp}$, so this is also a decomposition of $latex x$ with respect to $latex H = M^perp oplus M^{perpperp}$. However, $latex x$ already has a decomposition $latex x = x + 0$ with respect to $latex H = M^perp oplus M^{perpperp}$. The uniqueness clause in Theorem 12 now implies that $latex x = y in M$.
3. Projections with respect to orthonormal bases I
We know from a course in linear algebra that every finite dimensional inner product space has an orthonormal basis (see also the appendix). Let $latex M$ be a finite dimensional subspace of a Hilbert space $latex H$.
Exercise G: Prove that a finite dimensional subspace of an inner product space is closed.
Let $latex {e_n}_{n=1}^N$ be an orthonormal basis for $latex M$.
Theorem 15: With the above notation, for all $latex x in H$,
$latex P_M x = sum_{n=1}^N (x,e_n) e_n. $
Proof: Put $latex y = P_M x$. Since $latex y in M$, then $latex y = sum_{n=1}^N (y,e_n)e_n$. (The familiar proof from the course in linear algebra goes as follows:
$latex y = sum_{n=1}^N c_n e_n$
for some constants $latex c_n$, and taking the inner product of this equality with $latex e_k$ one obtains
$latex (y,e_k) = sum_{n=1}^N c_n (e_n, e_k) = c_k , $
and the representation of $latex y$ is as we claimed. By the way, this shows that every orthonormal set is linearly independent). But by Corollary 11, $latex (x-y, e_n) = 0 $, or $latex (x, e_n) = (y,e_n)$, for all $latex n$, therefore $latex y = sum_{n=1}^N (x,e_n) e_n$ , as asserted.
4. Orthonormal bases
Recall that a Hamel basis for a vector space $latex V$ is a family $latex {v_i}_{i in I}$ such that every $latex v in V$ can be written in a unique way as a (finite) linear combination of the $latex v_i$s. A vector space is said to be infinite dimensional if it has no finite Hamel basis. In linear algebra, a Hamel basis is called simply a “basis”, since every kind of basis considered in the finite dimensional case is a Hamel basis.
Exercise H: Prove that if $latex H$ is an infinite dimensional Hilbert space, then $latex H$ has no countable Hamel basis.
Speaking a little bluntly, the above exercise shows that Hamel bases are totally useless in infinite dimensional Hilbert spaces. We need another notion of basis.
Definition 16: Let $latex E = {e_i}_{i in I}$ be an orthonormal system in an inner product space $latex G$. $latex E$ is said to be complete if $latex E^perp = {0}$.
Proposition 17: Every inner product space has a complete orthonormal system.
Proof: One considers the set of all orthonormal systems in the space, ordered by inclusion, and applies Zorn’s Lemma to deduce the existence of a maximal orthonormal system. A maximal orthonormal system must be complete, otherwise one could add a normalized perpendicular vector.
In case that the inner product space in question is separable, one can also prove that there exists a complete orthonormal system by applying the Gram-Schmidt process (see the appendix) to a dense sequence.
Definition 18: Let $latex {e_i}_{i in I}$ be an orthonormal system in an inner product space $latex G$. For every $latex x in G$, the scalars $latex (x,e_i)$ are called the (generalized) Fourier coefficients of $latex x$ with respect to $latex {e_i}_{i in I}$.
By Bessel’s inequality (Proposition 5) and Exercise B, for every $latex x$, only coutably many Fourier coefficients are non-zero. This fact frees us, in the following proposition, to consider only countable orthonormal systems.
Proposition 19: Let $latex {e_n}_{n=1}^infty$ be an orthonormal system in an inner product space $latex G$. Then the following are equivalent:
- $latex sum_{n=1}^infty |(x,e_n)|^2 = |x|^2$.
- $latex sum_{n=1}^infty (x,e_n)e_n = x$.
- For all $latex epsilon > 0$, there exists an integer $latex N_0$ and scalars $latex a_1, ldots, a_{N_0}$ such that $latex |x – sum_{n=1}^{N_0} a_n e_n | < epsilon$.
Remark: The convergence of the series of vectors in 2 is to be interpreted simply as the assertion that $latex lim |sum_{n=1}^N (x,e_n) e_n – x | = 0$. Note that the equivalence 1 $latex Leftrightarrow$ 2 implies that this vector valued series converges regardless of the order in which it is summed, and also that this series also converges in the sense of Definition 4*.
Proof: As in the proof of Bessel’s inequality (Proposition 5) we find that
$latex |x|^2 = sum_{n=1}^N |(x,e_n)|^2 + |x – sum_{n=1}^N (x,e_n)e_n|^2 ,$
and this implies that 1 and 2 are equivalent. 2 obviously implies 3, because one simply takes $latex a_n = (x,e_n)$.
Assume that 3 holds. Let $latex epsilon > 0$ be given. We need to find $latex N_0$, such that for all $latex N geq N_0$, $latex |sum_{n=1}^N (x,e_n)e_n – x | < epsilon$. Let $latex N_0$ be the $latex N_0$ from 3 corresponding to $latex epsilon$, and let $latex a_1,ldots, a_{N_0}$ be the corresponding scalars. For any $latex N geq N_0$, the linear combination $latex sum_{n=1}^{N_0} a_n e_n$ is in the subpace spanned by $latex e_1, ldots, e_N$, which we denote by $latex M$. But by Theorem 15, $latex P_M x = sum_{n=1}^N (x,e_n) e_n$ is the best approximation for $latex x$ within $latex M$, therefore
$latex |sum_{n=1}^N (x,e_n) e_n – x| leq |sum_{n=1}^{N_0} a_n e_n – x| < epsilon .$
Proposition 20: Let $latex {e_i}_{i in I}$ be an orthonormal system in a Hilbert space $latex H$ and let $latex {a_i}_{i in I}$ be a set of complex numbers. The series $latex sum_{i in I}a_i e_i$ converges in $latex H$ if and only if $latex sum_{i in I}|a_i|^2 < infty$.
Proof: Suppose that $latex sum_{i in I}a_i e_i$ converges to some $latex x in H$. By Exercise B, there is a countable subset of $latex I$, say $latex J = {j_1, j_2, ldots, }$, such that $latex a_i = 0$ if $latex i notin J$ and such that
$latex lim_{N rightarrow infty} sum_{n=1}^N a_{j_n} e_{j_n} = x .$
Taking the inner product of the above with $latex e_i$, we find that $latex a_i = (x, e_i)$ for all $latex i in I$. Proposition 19 now tells us that $latex sum_{n=1}^infty |a_{j_n}|^2< infty$, therefore $latex sum_{i in I}|a_i|^2 < infty$.
Conversely, assume that $latex sum_{i in I}|a_i|^2 < infty$. Let $latex J$ be a countable index set as above. Define $latex x_N = sum_{n=1}^N a_n e_{j_n}$. Then it is easy to see that $latex {x_N}$ is a Cauchy sequence. Let $latex x$ be the limit of this sequence. Then continuity of the inner product implies that $latex (x, e_i) = a_i$ for all $latex i in I$. So we have $latex x = lim_{Nrightarrow infty} sum_{n=1}^N (x, e_{j_n}) e_{j_n}$. By the remark following Proposition 19,
$latex x = sum_{j in J} (x, e_{j}) e_{j} = sum_{i in I}a_i e_i ,$
and that completes the proof.
Theorem 21: Let $latex {e_i}_{i in I}$ be a complete orthonormal system in a Hilbert space $latex H$. Then for every $latex x in H$ the following hold:
- $latex x = sum_{i in I} (x, e_i) e_i$.
- $latex |x|^2 = sum_{i in I} |(x, e_i)|^2$.
Remark: There are two ways to interpret the (possibly uncountable) series in 1. One way is as in Definition 4*. However, since we know that for any $latex x$ only countably many Fourier coefficients are nonzero, we can interpret this series as in Proposition 19. Both approaches turn out to be equivalent (but only when summing orthonormal sequences).
Proof: Fix $latex x in H$. By Bessel’s inequality, $latex sum_{i in I}|(x,e_i)|^2 < infty$. By Proposition 20, the series $latex sum_{i in I} (x,e_i) e_i$ converges. Put $latex y = x – sum_{i in I} (x,e_i) e_i$. Our goal is to show that $latex y = 0$. Since $latex {e_i}_{i in I}$ is complete, it suffices to show that $latex (y, e_k) = 0$ for all $latex k in I$. By continuity of the inner product,
$latex (sum_{iin I}(x,e_i)e_i, e_k) = (x,e_k) (e_k, e_k) = (x, e_k)$,
thus $latex (y, e_k) = (x,e_k) – (x, e_k) = 0$ for all $latex k$. Thus $latex y perp {e_i}_{i in I}$, so $latex y = 0 $. By Proposition 19 assertions 1 and 2 are equivalent, thus the proof is complete.
Question: What role precisely does the completeness of the space play?
Because of the above theorem, a complete orthonormal system in a Hilbert space is often called an orthonormal basis. I think that this is highly justified terminology, and we will use it. An othonormal system (not necessarily in a Hilbert space) satisfying the conclusions of Theorem 21 is sometimes said to be a closed system. Perhaps this is to avoid the usage of the word “basis”, since an orthonormal basis is definitely not a basis according to the definition given in linear algebra (see Exercise H).
Remark: Assertion 2 in Theorem 21 is called Parseval’s identity.
Corollary 22 (Generalized Parseval’s identity): Let $latex {e_i}_{i in I}$ be an orthonormal basis for a Hilbert space $latex H$, and let $latex x, y in H$. Then
$latex (x,y) = sum_{i in I} (x,e_n) overline{(y,e_n)} .$
Proof: One uses Parseval’s identity together with the polarization identity
$latex (g,h) = frac{1}{4}left( |x+y|^2 – |x-y|^2 + i |x + iy|^2 – i|x-iy|^2 right) ,$
which holds in $latex H$ as well as in $latex mathbb{C}$.
5. Projections with respect to orthonormal bases II
Theorem 23: Let $latex M$ be a closed subspace in a Hilbert space $latex H$. Let $latex {e_i}_{i in I}$ be an orthonormal basis for $latex M$. Then for every $latex x in H$,
$latex P_M x = sum_{i in I}(x,e_i) e_i .$
Exercise I: Prove Theorem 22.
6. Dimension and isomorphism
Theorem 24: Let $latex {e_i}_{i in I}$ and $latex {f_j }_{j in J}$ be two orthonormal bases for the same Hilbert space $latex H$. Then $latex I$ and $latex J$ have the same cardinality.
Proof: If one of the index sets is finite then this result follows from linear algebra. So assume both sets are infinite. For every $latex i in I$, let $latex A_i = { j in J : (e_i, f_j) neq 0}$. Every $latex j in J$ belongs to at least one $latex A_i$, because $latex {e_i}_{i in I}$ is complete. Therefore $latex J = cup_{i in I} A_i$. But as we noted several times, $latex |A_i| leq aleph_0$. These two facts combine to show that the cardinality of $latex J$ is less than or equal to the cardinality of $latex I$. Reversing the roles of $latex I$ and $latex J$ we see that they must have equal cardinality.
Definition 25: Let $latex H$ be a Hilbert space. The dimension of $latex H$ is defined to be the cardinality of any orthonormal basis for $latex H$.
Definition 26: (a) Let $latex G_1,G_2$ be inner product spaces. A unitary map (or simply a unitary) from $latex G_1$ to $latex G_2$ is a bijective linear map $latex U : G_1 rightarrow G_2$ such that $latex (Ug, Uh)_{G_2} = (g,h)_{G_1}$ for all $latex g,h in G_1$. (b) Two inner product spaces $latex G_1,G_2$ are said to be isomorphic if there exists a unitary between them.
Theorem 27: Two Hilbert spaces are isomorphic if and only if they have the same dimension.
Exercise J: Prove Theorem 27.
Exercise K: Prove that a Hilbert space is separable if and only if its dimension is $latex aleph_0$ (recall that a metric space is said to be separable if it contains a countable dense subspace).
We have gone through some efforts to treat Hilbert spaces which are of arbitrary dimension. However, in mathematical practice, one rarely encounters (or wishes to encounter) a non-separable space. Nevertheless, rare things do happen. Occasionally it is useful to have an arbitrarily large Hilbert space for some universal construction to be carried out (for example, one needs this when proving that every C*-algebra is an algebra of operators on some Hilbert space). There are also some natural examples which arise in analysis.
Exercise L: Let $latex G$ be the linear span of all functions of the form $latex sum_{k=1}^n a_k e^{i lambda_k t}$. On $latex G$, we define a form
$latex (f,g) = lim_{T rightarrow infty} frac{1}{2T}int_{-T}^T f(t) overline{g(t)} dt .$
Prove that $latex G$ is an inner product space. Let $latex H$ be the completion of $latex G$. Prove that $latex H$ is not separable. Find an orthonormal basis for $latex H$.
Appendix: Gram-Schmidt process
We now give a version of the Gram-Schmidt orthogonalization process appropriate for sequences of vectors. In the case where the sequence is finite, this is precisely the same procedure studied in a course in linear algebra.
Theorem 28: Let $latex v_1, v_2, ldots$ be a sequence of vectors in an inner product space $latex G$. Then there exists an orthonormal sequence $latex e_1, e_2, ldots$ with the same linear span as $latex v_1, v_2, ldots$. If the sequence $latex v_1, v_2, ldots$ is linearly independent, then for all $latex n=1, 2, ldots $,
$latex textrm{span}{e_1, ldots, e_n} = textrm{span}{v_1, ldots, v_n}.$
Proof: From every sequence of vectors one can extract a linearly independent sequence, so it suffices to prove the second half of the theorem. Assume that $latex v_1, v_2, ldots$ is a linearly independent sequence. We prove the claim by induction on $latex n$. For $latex n=1$ we put $latex e_1 = v_1 / |v_1|$. Assume that $latex n > 1$, and that we have constructed an orthonormal sequence $latex e_1, ldots, e_{n-1}$ such that
$latex textrm{span}{e_1, ldots, e_{n-1}} = textrm{span}{v_1, ldots, v_{n-1}}.$
Let $latex M$ denote the subspace appearing in the above equality, and let $latex P_M$ be the orthogonal projection onto $latex M$. Then $latex v_n$ is not in $latex M$. Put $latex u = v_n – P_M v_n$. Then $latex u neq 0$, and by Corollary 11, $latex u in M^perp$. Let $latex e_n = u/|u|$. Then $latex e_1, ldots, e_n$ is an orthonormal sequence, and $latex e_n in textrm{span}{v_1, ldots, v_n}$ by construction. Thus $latex textrm{span}{e_1, ldots, e_{n}} subseteq textrm{span}{v_1, ldots, v_{n}}$. But since $latex e_1 ldots, e_n$ are linearly independent (as any orthonormal set; see the proof of Theorem 15), and there $latex n$ of them, we must have $latex textrm{span}{e_1, ldots, e_n} = textrm{span}{v_1, ldots, v_n}$. That completes the proof.
The proof of theorem 15:
[
sum_{n=1} ^N (y,e_n) e_n
]
should be replaced by
[
y=sum_{n=1} ^N (y,e_n) e_n
]
Thanks Alon, corrected