Advanced Analysis, Notes 16: Hilbert function spaces (basics)

In the final week of the semester we will study Hilbert function spaces (also known as reproducing kernel Hilbert spaces) with the goal of presenting an operator theoretic proof of the classical Pick interpolation theorem. Since time is limited I will present a somewhat unorthodox route, and ignore much of the beautiful function theory involved. BGU students who wish to learn more about this should consider taking Daniel Alpay’s course next semester. Let me also note the helpful lecture notes available from Vern Paulsen’s webpage and also this monograph by Jim Agler and John McCarthy (in this post and the next one I will refer to these as [P] and [AM] below).

(Not directly related to this post, but might be of some interest to students: there is an amusing discussion connected to earlier material in the course (convergence of Fourier series) here).

1. Hilbert function spaces

A Hilbert function space on a set $latex X$ is a Hilbert space $latex H$ which is a subspace of the vector space of all functions $latex X rightarrow mathbb{C}$, with one additional and crucial property:

For every $latex x in X$ the functional of point evalution at $latex x$, given by $latex f mapsto f(x)$, is a bounded functional on $latex H$.

This crucial property connects between the function theoretic properties of $latex H$ as a space of functions, and the Hilbert space structure of $latex H$ and therefore also the operator theory on $latex H$. Hilbert function spaces are also known as reproducing kernel Hilbert spaces (RKHS), for reasons that will become clear very soon.

When we say that $latex H$ is a subspace of $latex mathbb{C}^X$, we mean two things: 1) $latex H$ is a subset of $latex mathbb{C}^X$; and 2) $latex H$ inherits its linear space structure from $latex mathbb{C}^X$. In particular, the zero function is the zero element of $latex H$, and two elements of $latex H$ which are equal as functions are equal as elements of $latex H$.

Example: Perhaps the most interesting examples of Hilbert function spaces are those that occur when $latex X$ is an open subset of $latex mathbb{C}^N$, and now we’ll meet one of the most beloved examples. Let D denote the open unit disc in the complex plane $latex mathbb{C}$. Recall that every function $latex f$ analytic in D has a power series representation $latex f(z) = sum_{n=0}^infty a_n z^n$. We define the Hardy space $latex H^2$ to be the space of analytic functions on the disc with square summable Taylor coefficients, that is

$latex H^2 = {f(z) = sum_{n=0}^infty a_n z^n : sum|a_n|^2 < infty }.$

This space is endowed with the inner product

(*)  $latex langle sum a_n z^n , sum b_n z^n rangle = sum a_n overline{b_n}.$

Before worrying about function theoretic issues, note that the space of formal power series with square summable coefficients is obviously isomorphic (as an inner product space) to $latex ell^2$ when endowed with the inner product (*). But if a formal power series has square summable coefficients then its coefficients are bounded, so it is convergent in the open unit disc. Thus $latex H^2$ is a space of analytic functions which as an inner product space is isomorphic to $latex ell^2$, and is therefore complete.

To show that $latex H^2$ is a Hilbert function space on D, we need to show that for every $latex w in D$, the functional $latex f mapsto f(w)$ is bounded. Well, if this functional is bounded, then by the Riesz representation theorem there exists some $latex k_w in H^2$ such that $latex f(w) = langle f, k_w rangle$. Take another look at (*), recall that $latex f(w) = sum a_n w^n$, and putting these two things together it is not hard to come up with the guess that $latex k_w(z) = sum_{n=0}^infty overline{w}^n z^n$. Since $latex |w|<1$, we have that $latex sum |overline{w}^n|^2 < infty$, so $latex k_w in H^2$. Plugging $latex k_w$ in (*) we see that $latex langle f, k_w rangle = f(w)$, so that this functional is indeed bounded.

There are a great many other Hilbert function spaces of rather different function theoretic nature. Some of them are mentioned and studied in [P] and in [AM].

Example: The Hilbert space $latex ell^2 = ell^2(mathbb{N})$ is a space of functions from $latex mathbb{N}$ to $latex mathbb{C}$, and point evaluation is clearly bounded. Thus $latex ell^2$ is a Hilbert function space on set $latex mathbb{N}$. I have never heard of anything useful that came out of this point of view.

Exercise A.1: The Bergman space $latex L^2_a(D)$ is the space of all analytic functions in the disc which are square integrable on the disc:

$latex L^2_a(D) = L^2(D) cap O(D) $

(where $latex O(D)$ denotes the space of functions analytic in D). Show directly (that is, not by guessing the kernel functions) that point evaluation is bounded.

Exercise A.2: The Segal-Bargmann space is the space of all entire functions $latex f$ for which $latex int_mathbb{C} |f(x+iy)|^2 e^{-x^2-y^2}dxdy < infty$. Show that this space is a Hilbert function space.

2. Reproducing kernels

Let $latex H$ be a Hilbert function space on a set $latex X$. As point evaluation is a bounded functional, there is, for every $latex x in X$, an element $latex k_x in H$ such that $latex f(x) = langle f, k_x rangle$ for all $latex f in H$. The function $latex k_x$ is called the kernel function at $latex x$. We define a function on $latex X times X$ by

$latex k(x,y) = k_y(x) .$

The function $latex k$ is called the reproducing kernel $latex H$ or simply the kernel of $latex H$. The terminology comes from the fact that the family of functions $latex k_y = k(cdot, y)$ enable one to reproduce the values of any $latex f in H$ via the relationship $latex f(y) = langle f, k_y rangle$. Note that $latex k(x,y) = langle k_y, k_x rangle$.

Example: The kernel function for $latex H^2$ is

$latex k(z,w) = k_w(z) = sum overline{w}^n z^n = frac{1}{1-zoverline{w}} .$

Exercise B: What is the kernel function of $latex ell^2(mathbb{N})$? What about the Bergman space $latex L^2_a(D)$? What about the Segal-Bargmann space?

The kernel $latex k$ has the following important property. If $latex a_1, ldots, a_n in mathbb{C}$ and $latex x_1, ldots, x_n in X$, then

$latex sum_{i,j=1}^n a_i overline{a_j} k(x_j,x_i) = sum_{i,j=1}^n a_i overline{a_j} langle k_{x_i}, k_{x_j} rangle = |sum_{i=1}^n a_i k_{x_i}|^2 geq 0.$

It follows that for every choice of points $latex x_1, ldots, x_n in X$, the matrix $latex big(k(x_j, x_i) big)_{i,j=1}^n$ is positive semi-definite. A function with this property is called a positive definite kernel (sometimes it is called a positive semi-definite kernel, but let us lighten terminology). Every Hilbert function space gives rise to a positive definite kernel — its reproducing kernel. In fact, there is a bijective correspondence between positive definite kernels and Hilbert function spaces.

Theorem 1: Let $latex k$ be a positive definite kernel on $latex X$. Then there exists a unique Hilbert function space $latex H$ on $latex X$ such that $latex k$ is the reproducing kernel of $latex H$.

Remark: Let me be a little bit more precise regarding the uniqueness statement, because in the context of Hilbert space one might mistakingly understand that this uniqueness is up to some kind of Hilbert space isomorphism. The existence statement is that there exists a subset $latex H$ of the space $latex mathbb{C}^X$ of all functions on $latex X$, which is a Hilbert function space, and has $latex k$ as a reproducing kernel. The uniqueness assertion is that this set $latex H subseteq mathbb{C}^X$ is uniquely determined.

Proof: Denote $latex k_x = k(cdot, x)$. Let $latex G_0$ be the linear subspace of $latex mathbb{C}^X$ spanned by the functions $latex {k_x : x in X}$. Equip $latex G_0$ with the sesquilinear form

$latex langle sum a_i k_{x_i}, sum b_j k_{y_j} rangle_0 = sum a_i overline{b_j} k(y_j, x_i) .$

From the definition and the fact that $latex k$ is a positive definite kernel, the form $latex langle cdot, cdot rangle_0$ satisfies properties 2–4 from the definition of inner product (see Definition 1 in this post). That is, it is almost an inner product, the only property missing is $latex langle g, grangle = 0 Rightarrow g = 0$. As mentioned before (see Theorem 2 in that same old post) this means that the Cauchy-Schwartz inequality holds for this form on $latex G_0$. From this it follows that the space $latex N = {g in G_0 : langle g, grangle_0 = 0}$ is a subspace, and also that $latex langle f,g rangle_0 = 0$ for all $latex f in G_0$ and $latex g in N$. This implies that the quotient space $latex G = G_0 / N$ can be equipped with the inner product

$latex langle dot{f}, dot{g} rangle = langle f, g rangle_0 .$

As usual, we are using the notation $latex dot{f}$ to denote the equivalence class of $latex f$ in $latex G_0 / N$. That this is well defined and an inner product follows from what we said about $latex N$. Now complete $latex G$ to obtain a Hilbert space $latex H$.

We started with a space of functions on $latex X$, but by taking the quotient and completing we are now no longer dealing with functions on $latex X$. To fix this, we define a map $latex F: H rightarrow mathbb{C}^X$ by

(**) $latex F(h)(x) = langle h, dot{k}_x rangle .$

This is a linear map and it is injective because the set $latex {dot{k}_x : x in X}$ spans $latex G$. Note that

(***) $latex F(dot{k_x}) (y) = langle dot{k}_x, dot{k}_y rangle = k(y,x)$.

We may push the inner product from $latex H$ to $latex F(H)$, and we obtain a Hilbert space of functions in $latex mathbb{C}^X$. Point evaluation is bounded by definition (see (**)), and the kernel function at $latex x$ is $latex F(dot{k}_x)$.  By (***), this $latex F(dot{k}_x) = k_x$. Let us now identify between $latex H$ and $latex F(H)$. It follows that $latex H$ is spanned by the kernel functions $latex {k_x : x in X}$. The kernel function of this Hilbert function space $latex H$ is given by

$latex langle dot{k}_y, dot{k}_x rangle = langle k_y, k_x rangle = k(x,y) ,$

as required.

Now suppose that $latex L$ is another Hilbert function space on $latex X$ that has $latex k$ as a reproducing kernel. By definition, the kernel functions are contained in $latex L$, and since $latex H$ is spanned by the kernel functions, $latex H subseteq L$. However, if $latex f$ is in the orthogonal complement of $latex H$ in $latex L$ then $latex f(x) = langle f, k_x rangle = 0$ for all $latex x in X$, thus $latex f equiv 0$. This shows that $latex L = H$, completing the proof.

Remark: Sometimes Hilbert function spaces are referred to as reproducing kernel Hilbert spaces (RKHS for short), this terminology being justified by the preceding discussion.

Exercise C: Let $latex H$ be a RKHS with kernel $latex k$. Let $latex x_1, ldots, x_n in X$. Show that the matrix $latex big( k(x_i, x_j) big)_{i,j=1}^n$ is strictly positive definite (meaning that it is positive semi-definite and invertible) iff the functions $latex k_{x_1}, ldots, k_{x_n}$ are linearly indendent, iff the evaluation functionals at the points $latex x_1, ldots, x_n$ are linearly independent as functionals on $latex H$.

Exercise D: Show how the proof of the above theorem is simplified when $latex k$ is assumed strictly positive definite (meaning that the matrix $latex big( k(x_i, x_j) big)_{i,j=1}^n$ is strictly positive definite for any choice of points). Show how at every stage of the process one has a space of functions on $latex X$.

Exercise E: Rewrite the above proof of the theorem (for $latex k$ not necessarily strictly positive) so that it involves no quotient-taking and no abstract completions.

3. $latex H^2$ as a “subspace” of $latex L^2$

We now continue to study the space $latex H^2$, and give a useful formula for the norm of elements in this space.

Let $latex f(z) = sum_{n=0}^infty a_n z^n in H^2$. For any $latex rin (0,1)$ denote $latex f_r(z) = f(rz)$. The power series of $latex f_r$ is given by

$latex f_r(z) = sum_{n=0}^infty a_n r^n z^n .$

This series converges to $latex f_r$ uniformly on the closed unit disc. In fact, $latex f_r$ extends to an analytic function on a neighborhood of the closed unit disc, and the series converges uniformly and absolutely on a closed disc that is slightly larger than the unit disc. It follows that the following computations are valid:

$latex frac{1}{2pi}int_0^{2 pi} |f_r(e^{it})|^2 dt = sum_{m,n=0}^infty frac{1}{2 pi} int_0^{2 pi} a_m r^m e^{imt} overline{a_n} r^n e^{-int} dt =$

$latex = sum_{n=0}^infty r^{2n}|a_n|^2 = |f_r|^2_{H^2} .$

It would be nice to say that for every $latex f in H^2$

$latex |f|_{H^2} = big( frac{1}{2 pi} int_0^{2pi} |f(e^{it})|^2 dt big)^{1/2} ,$

and this is almost true, but we have to be careful because elements in $latex H^2$ are not defined on $latex mathbb{T} = {z in mathbb{C} : |z|=1}$. However,

$latex |f-f_r|^2 = sum (1-r^n)^2|a_n|^2 $

and this tends to $latex 0$ as $latex r rightarrow 1$ by the dominated convergence theorem. In particular

$latex |f|^2 = lim_{rrightarrow 1} |f_r|^2 = lim_{rrightarrow 1}frac{1}{2pi}int_0^{2 pi} |f(re^{it})|^2 dt .$

We will use this formula below. Much more is true in fact: the functions $latex f_r$ converge almost everywhere to a function $latex f^* in L^2(mathbb{T})$, and $latex |f|_{H^2} = |f^*|_{L^2}$. Moreover, $latex H^2$ can be identified via the map $latex f mapsto f^*$ with the subspace of $latex L^2(mathbb{T})$ consisting of all functions with Fourier series that are supported on the non-negative integers: if $latex f(z) = sum_{n=0}^infty a_n z^n in H^2$, then $latex f^*(e^{it}) = sum_{n=0}^infty a_n e^{int}$. For details, see [AM]. What we will need is a little weaker fact:

Fact: A function $latex f$ analytic on D is in $latex H^2$ if and only if

$latex lim_{rrightarrow 1} int_0^{2pi}|f(re^{it})|^2 dt < infty . $

Proof: We already showed that $latex H^2$ functions have this property. Now let $latex f$ be analytic in the disc, and suppose it has power series representation $latex f(z) = sum_{n=0}^infty a_n z^n$. As before

$latex frac{1}{2pi}int_0^{2pi}|f(re^{it})|^2 = sum_{n=0}^infty r^{2n}|a_n|^2 .$

By the monotone convergence theorem the right hand side converges to $latex sum_{n=0}^infty |a_n|^2$, so the limit of integrals is finite if and only if $latex f in H^2$.

4. Multiplier algebras

Let $latex H$ be a Hilbert function space on $latex X$, to be fixed in the following discussion. Now we define a class of natural operators on $latex H$.

The multiplier algebra of $latex H$, denoted $latex Mult(H)$, is defined as

$latex Mult(H) = {f : X rightarrow mathbb{C} : forall h in H . fh in H} .$

In this definition $latex fh$ simply denotes the usual pointwise product of two functions: $latex fh(x) = f(x) h(x)$. For every $latex f in Mult(H)$ we define an operator $latex M_f : H rightarrow H$ by

$latex M_f h = fh .$

It is not unusual to identify $latex M_f$ and $latex f$, and we will do so too, but sometimes it is helpful to distinguish between the function $latex f$ and the linear operator $latex M_f$.

Proposition 2: For all $latex f in Mult(H)$, we have $latex M_f in B(H)$.

Proof: Obviously $latex M_f$ is a well defined linear operator. To show that it is bounded we use the closed graph theorem. Let $latex h_n rightarrow h in H$, and suppose that $latex M_f h_n = f h_nrightarrow g in H$. Since point evaluation is continuous we have for all $latex x in X$

$latex h_n(x) rightarrow h(x)$

and

$latex f(x) h_n(x) rightarrow g(x)$

thus $latex f h(x) = g(x)$ for all $latex x$, meaning that $latex M_f h = g$, so the graph of $latex M_f$ is closed.

Thus, $latex {M_f : f in Mult(H)}$ is a subalgebra of $latex B(H)$. We norm $latex Mult(H)$ by pulling back the norm from $latex B(H)$ (this means $latex |f|_{Mult(H)}:= |M_f|$), and this turns $latex Mult(H)$ into a normed algebra. We will see below that it is a Banach algebra.

Important assumption: In fact, there is a situation when $latex |f|_{Mult(H)}$ is not a norm, but only a semi-norm. This doesn’t arise when for every $latex x in X$ there exists a function $latex h in H$ such that $latex h(x) neq 0$. Thus we will assume this henceforth. This assumption is holds in most (but not all) cases of interest, and in particular in the case of $latex H^2(D)$. It is equivalent to the assumption that $latex k_x(x) = k(x,x) = |k_x|^2 neq 0$ for all $latex x in X$.

One of the instances when it is helpful to distinguish between a multiplier $latex f$ and the multiplication operator $latex M_f$ that it induces is when discussing adjoints.

Proposition 3: Let $latex fin Mult(H)$. For all $latex x in X$, the kernel function $latex k_x$ is an eigenvector for $latex M_f^*$ with eigenvalue $latex overline{f(x)}$. Conversely, if $latex T in B(H)$ is an operator such that $latex T k_x = lambda(x) k_x$ for all $latex x in X$, then there is an $latex f in Mult(H)$ such that $latex lambda(x) = overline{f(x)}$ for all $latex x$ and $latex T = M_f^*$.

Proof: We let $latex x in X$ and compute for all $latex h in H$

$latex f(x)h(x) = langle M_f h, k_x rangle = langle h, M_f^* k_x rangle$

but $latex f(x) h(x) = langle h, overline{f(x)} k_x rangle$. Thus $latex M_f^* k_x = overline{f(x)} k_x$.

For the converse, suppose $latex T k_x = lambda(x) k_x$ for all $latex x in X$. Define $latex f(x) = overline{lambda(x)}$. For all $latex h in H$, we have $latex T^*h in H$ too. We now compute

$latex langle T^* h , k_x rangle = langle h, T k_x rangle = overline{lambda(x)} langle h, k_x rangle = f(x) h(x).$

It follows that $latex T^* h = fh in H$, so $latex f in Mult(H)$. It also follows that $latex T^* = M_f$.

Corollary 4: $latex Mult(H)$ contains only bounded functions, and $latex sup_{X}|f| leq |M_f|$.

Proof: This follows from the Propositions 2 and 3, since the eigenvalues of $latex M_f^*$ are contained in $latex sigma(M_f^*)$, which is contained in the disc of radius $latex |M_f^*| = |M_f|$.

Corollary 5: $latex {M_f : f in Mult(H)}$ is closed in the weak operator topology of $latex B(H)$, and, in particular, $latex Mult(H)$ is a Banach algebra.

Remark: The weak operator topology is the topology on $latex B(H)$ in which a net $latex T_n rightarrow T$ if and only for all $latex f,g in H$, $latex langle T_n f, g rangle rightarrow langle Tf, g rangle$. The reader may safely replace this with the norm topology of $latex B(H)$ for the purposes of these lectures.

Proof: If $latex T_n$ is a net in $latex {M_f : f in Mult(H)}$ converging to $latex T$ in the weak operator topology, then $latex T_n^* rightarrow T^*$ WOT. Then for all $latex x in X$, $latex T_n^* k_x = lambda_n(x) k_x$ converges weakly to $latex T^* k_x$, so $latex k_x$ is an eigenvalue for $latex T^*$. By Proposition 3, $latex T in {M_f : f in Mult(H)}$.

Example: Let’s find out who is the multiplier algebra of $latex H^2$. Since $latex 1 in H^2$, we have that $latex Mult(H) subseteq H^2$, and in particular every multiplier in $latex Mult(H)$ is an analytic function on D. By Corollary 4 we have that every multiplier is bounded on the disc. Thus

$latex Mult(H^2) subseteq H^infty equiv H^infty(D) .$

The right hand side denotes the algebra of bounded analytic functions in D. On the other hand, by the formula developed in Section 3, we have for $latex f in H^infty$ and $latex h in H^2$,

$latex lim_{rrightarrow 1}frac{1}{2 pi}int_0^{2pi} |f(re^{it})h(re^{it})|^2 dt leq |f|^2_infty lim_{rrightarrow 1} frac{1}{2pi} int_0^{2pi} |h(re^{it})|^2 dt = |f|^2_infty |h|^2_{H^2} .$

By the fact at the end of Section 3, we obtain $latex fh in H^2$ and $latex |fh|_{H^2} leq |f|_infty |h|_{H^2}$, so $latex f in Mult(H^2)$ and $latex |M_f| leq |f|_infty$.

Exercise F: What is the the multiplier algebra of $latex L^2_a(D)$? Of $latex ell^2(mathbb{N})$? Can you cook up a non-trivial RKHS that has a trivial (scalars only) multiplier algebra?

5. First magic trick

Note that as a consequence of the previous example and Corollary 5 we have that $latex H^infty(D)$ is a Banach algebra. It follows readily that the limit of any locally uniformly convergent sequence of analytic functions (on any open set) is also analytic. I leave it to the reader to check just how much complex analysis we used to obtain this result, and to compare with the other proof they know of this fact.