Geometry of Hilbert Spaces

Definition. If $\mathcal{M}$ is a subset of the Hilbert space $\mathscr{H}$, then the orthogonal complement of $\mathcal{M}$, denoted $\mathcal{M}^{\perp}$,is the set of vectors in $\mathscr{H}$ orthogonal every vector in $\mathcal{M}$.

Remark. $\mathcal{M}^{\perp}$ is a closed subspaces of $\mathscr{H}$.

Proof. Let $x$,$y$ $\in \mathscr{M}^{\perp}$, $\forall z \in \mathcal{M}$, $<{z},{x}> = <{z},{y}> = 0$. Then,

$<{z},{x+y}> = <{z},{x}> + <{z},{y}> = 0+0 = 0.$

This implies $x+y \in \mathscr{M}^{\perp}$. And $\forall \lambda \in \mathbb{C}$, $<{\lambda z},{x}> = 0$, then $\lambda z \in \mathscr{M}^{\perp}$. Hence, $\mathscr{M}^{\perp}$ is a subspace of $\mathscr{H}$.

Next, we only need to show $\mathscr{M}^{\perp}$ is closed. For $x \in \mathcal{M}$, we can define a linear functional $\varphi_x(z) := <{z},{x}>$, them$\mathscr{M}^{\perp} = \bigcap_{x \in \mathcal{M}} \textbf{Ker}{\varphi_{x}}$ is a closed subset of $\mathscr{H}$. ◻

Theorem. If $\mathcal{M}$ is a closed subspace of the Hilbert space $\mathscr{H}$ and $f \in \mathscr{H}$, then there exist unique vectors $g \in \mathcal{M}$ and $h \in \mathscr{M}^{\perp}$ such that

$f = g + h.$

Proof. Set $\mathscr{K} = \{f - k: k \in \mathcal{M}\}$, then $\mathscr{K}$ is a nonempty, closed and convex subset of $\mathscr{H}$.
Hence, there exists the unique element $h \in \mathscr{K}$ with the smallest norm. Consider $k \in \mathcal{M}$, then $h - <{h},{k}>k \in \mathscr{K}$, and hence

$\begin{aligned} {\|{h}\|}^2 &\le {\|{h - <{h},{k}>k}{}\|}^2 \\ &= \|{h}{}\|^2 - \overline{<{h},{k}>} <{h},{k}> - <{h},{k}>\overline{<{h},{k}>} + <{h},{k}>\overline{<{h},{k}>}{\|{k}{}\|}^2. \end{aligned}$

Therefore, we have $|{<{h},{k}>}|^2 \le 0$, which implies $<{h},{k}> = 0$, $h\in \mathscr{M}^{\perp}$. Since $h \in \mathscr{K}$, then $\exists g \in \mathcal{M}$, such that $f = g+h$.

Uniqueness: If $f = g_1 + h_1 = g_2 + h_2$, with $g_1, g_2 \in \mathcal{M}$, and $h_1, h_2 \in \mathscr{M}^{\perp}$. Then $g_1 - g_2 = h_1 - h_2 \in \mathcal{M} \bigcap \mathscr{M}^{\perp}$. This means that $(g_1 - g_2) \perp (g_1 - g_2)$, and $(h_1 - h_2) \perp (h_1 - h_2)$. $<{g_1 - g_2},{g_1 - g_2}> = <{h_1 - h_2}, {h_1 - h_2}> = \|{g_1 - g_2}{}\|^2 = \|{h_1 - h_2}{}\|^2 = 0.$
Then $g_1 = g_2$ and $h_1 = h_2$. ◻

Corollory. If $\mathcal{M}$ is a subspace of the Hilbert space $\mathscr{H}$, then $\mathcal{M}^{\perp \perp} = \overline{\mathcal{M}}$.

Proof. Obvious that $\mathcal{M} \subset \mathcal{M}^{\perp \perp}$(By definition of $\mathscr{M}^{\perp}$). Hence, $\overline{\mathcal{M}} \subset \mathcal{M}^{\perp \perp}$.
If$f \in \mathcal{M}^{\perp \perp}$, then $f = g + h$, where $g \in \overline{\mathcal{M}}$, and $h \in \mathscr{M}^{\perp}$. Since $f \in \mathcal{M}^{\perp \perp}$, we have

$0 = <{f},{h}> = <{g+h},{h}> = <{h},{h}> = \|{h}{}\|^2.$

Therefore, $h = 0$, hence, $f \in \overline{M}$. ◻

Theorem. If $\varphi$ is a bounded linear functional on $\mathscr{H}$, then there
exists a unique $g \in \mathscr{H}$ such that $\varphi(f) = <{f},{g}>$ for $f \in \mathscr{H}$.

Proof. Let $\mathscr{K} = \textbf{Ker} \varphi = \{f \in \mathscr{H} : \varphi(f) = 0\}$. Since $\varphi$ is continuous, $\mathscr{K}$ is a closed subspace of $\mathscr{H}$. If $\mathscr{K} = \mathscr{H}$, then $\varphi(f) = <{f},{0}>$, $\forall f \in \mathscr{H}$.

If $\mathscr{K} \subsetneq \mathscr{H}$, then $\exists h \in \mathscr{K}^{\perp}$ with $\|{h}{}\| = 1$. Since $h \notin \mathscr{K}$, then $\varphi(h) \neq 0$.

Since

$\varphi(f - \frac{\varphi(f)}{\varphi(h)} h) = \varphi(f) - \frac{\varphi(f)}{\varphi(h)}\varphi(h) = 0, \quad \forall f \in \mathscr{H}.$

Then $\forall f \in \mathscr{H}$, $f - \frac{\varphi(f)}{\varphi(h)} h \in \mathscr{K}$. Therefore, for
$f\in \mathscr{H}$, we have

$\begin{aligned} \varphi(f) &= \varphi(f)<{h},{h}> = <{\frac{\varphi(f)}{\varphi(h)} h },{\overline{\varphi(h)} h }> \\ &= <{\frac{\varphi(f)}{\varphi(h)} h },{\overline{\varphi(h)} h }> + <{f - \frac{\varphi(f)}{\varphi(h)} h},{\overline{\varphi(h)} h}> \\ &= <{f},{\overline{\varphi(h)}h}>. \end{aligned}$

Uniqueness: If $<{f},{g_1}> = <{f},{g_2}>$
$\forall f \in \mathscr{H}$, then $<{g_1 - g_2},{g_1 - g_2}> = 0$,
$g_1 = g_2$. ◻

Theorem. If $\{f_{\alpha}\}_{\alpha \in \Lambda}$ is an orthogonal subset of Hilbert space $\mathscr{H}$, then $\sum_{\alpha \in \Lambda} f_{\alpha}$ converges if and only if $\sum_{\alpha \in \Lambda} \|{f_{\alpha}}{}\|^2 < +\infty$. And in this case, we have

$\|{\sum_{\alpha \in \Lambda} f_{\alpha}}{}\|^2 = \sum_{\alpha \in \Lambda}\|{f_{\alpha}}{}\|^2.$

Back to the definition of arbitrary sum:

Definition. Let $\{f_{\alpha}\}_{\alpha \in A}$ be a set of vectors in the Banache space $X$. Let $\mathscr{F} = \{F \subset A : F \text{ is finite}\}$. If we define $F_1 \le F_2$ for $F_1 \subset F_2$, then $\mathscr{F}$ is a directed set(also means the partial ordered set). For each $F \in \mathscr{F}$, let $g_F = \sum_{\alpha \in F} f_{\alpha}$. If the net $\{g_F\}_{F \in \mathscr{F}}$ converges to some $g$ in $X$, then the sum $\sum_{\alpha \in A} f_{\alpha}$ is said to converge and we write $g = \sum_{\alpha \in A}f_{\alpha}$.

Proof. Consider a finite subset $F \subset \Lambda$, then, we have

$\|{\sum_{\alpha \in F} f_{\alpha}}{}\|^2 = \sum_{\alpha \in F} \|{f_{\alpha}}{}\|^2.$

Let $\mathscr{F}$ denote the collection of finite subsets of $\Lambda$. If $\sum_{\alpha \in \Lambda} f_{\alpha}$ converges, then

$\begin{aligned} \|{\sum_{\alpha \in \Lambda} f_{\alpha}}{}\|^2 &= \|{\lim_{F \in \mathscr{F}} \sum_{\alpha \in F} f_{\alpha}}{}\|^2 = \lim_{F \in \mathscr{F}} \|{\sum_{\alpha \in F} f_{\alpha}}{}\|^2 \\ &= \lim_{F \in \mathscr{F}}\sum_{\alpha \in F} \|{f_{\alpha}}{}\|^2 = \sum_{\alpha \in \Lambda} \|{f_{\alpha}}{}\|^2. \end{aligned}$

Hence, if $\sum_{\alpha \in \Lambda} f_{\alpha}$ convergers, then $\sum_{\alpha \in \Lambda} \|{f_{\alpha}}{}\|^2 < \infty$.

Conversely, if $\sum_{\alpha \in \Lambda} \|{f_{\alpha}}{}\|^2 < \infty$. $\forall \varepsilon > 0$, $\exists F_0 \in \mathscr{F}$ such that $F \ge F_0$ implies $\sum_{\alpha \in F} \|{f_{\alpha}}{}\|^2 - \sum_{\alpha \in F_0} \|{f_{\alpha}}{}\|^2 < \varepsilon^2$.
Thus, for $F_1,F_2 \in \mathscr{F}$ such that $F_1, F_2 \ge F_0$, we have

$\begin{aligned} \|{\sum_{\alpha \in F_1} f_{\alpha} - \sum_{\alpha \in F_2} f_{\alpha}}{}\|^2 &= \sum_{\alpha \in F_1 \backslash F_2} \|{f_{\alpha}}{}\|^2 + \sum_{\alpha \in F_2 \backslash F_1 } \|{f_{\alpha}}{}\|^2 \\ &\le \sum_{\alpha \in F_1 \cup F_2} \|{f_{\alpha}}{}\|^2 - \sum_{\alpha \in F_0} \|{f_{\alpha}}{}\|^2 < \varepsilon^2. \end{aligned}$

Hence, the net $\{\sum_{\alpha \in F} f_{\alpha}\}_{F \in \mathscr{F}}$ is Cauchy in $\mathscr{H}$, which is convergence. ◻

Corollary. If $\{e_{\alpha}\}_{\alpha \in A}$ is an orthonormal subset of the Hilbert space $\mathscr{H}$ and $\mathcal{M}$ is the smallest closed subspace of $\mathscr{H}$ containing the set $\{e_{\alpha} : \alpha \in A\}$, then

$\mathcal{M} = \left\{\sum_{\alpha \in A} \lambda_{\alpha} e_{\alpha }: \lambda_{\alpha} \in \mathbb{C} , \sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 < \infty\right\}.$

Proof. Let $\mathcal{N} = \left\{\sum_{\alpha \in A} \lambda_{\alpha} e_{\alpha }: \lambda_{\alpha} \in \mathbb{C} , \sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 < \infty\right\}$. We have $\mathcal{N} \subset \mathcal{M}$.
Then, we only need to show $\mathcal{N}$ is a closed subspace of $\mathscr{H}$ containing the set $\{e_{\alpha}\}_{\alpha \in A}$.

Step 1. We prove that $\mathcal{N}$ is a closed subspace of $\mathscr{H}$ containing the set $\{e_{\alpha} : \alpha \in A\}$.

Assume $\{\lambda_{\alpha}\}_{\alpha \in A}, \{\mu_{\alpha}\}_{\alpha \in A}$ satisfy $\sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 < \infty$ and $\sum_{\alpha \in A } |{\mu_{\alpha}}|^2 < \infty$, then

$\begin{aligned} \sum_{\alpha \in A} |{\lambda_{\alpha}+ \mu_{\alpha}}|^2 &\le \sum_{\alpha \in A} 2(|{\lambda_{\alpha}}|^2 + |{\mu_{\alpha}}|^2)\\ &= 2\sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 + 2 \sum_{\alpha \in A} |{\mu_{\alpha}}|^2 < \infty. \end{aligned}$

This implies $\sum_{\alpha \in A} \lambda_{\alpha}e_{\alpha} + \sum_{\alpha \in A} \mu_{\alpha}e_{\alpha} = \sum_{\alpha \in A}(\lambda_{\alpha} + \mu_{\alpha})e_{\alpha} \in \mathcal{N}$. and it is easy to verify that $\sum_{\alpha \in A} a \lambda_{\alpha}e_{\alpha} \in \mathcal{N}$ for all $a \in \mathbb{C}$ and $e_{\alpha} \in \mathcal{N}, \forall \alpha \in A$.

Step 2. We prove that $\mathcal{N}$ is closed and $\mathcal{N} \subset \mathcal{M}$.

Note that $\mathcal{N} = \ell^2(A)$ actually. Then $\mathcal{N}$ is closed. ◻

Definition. A subset $\{e_{\alpha}\}_{\alpha \in A}$ of the Hilbert space $\mathscr{H}$ is said to be an orthonormal basis if it is orthonormal and the smallest closed subspace containing it is $\mathscr{H}$.

Corollary. If $\{e_{\alpha}\}_{\alpha \in A}$ is an orthonormal basis for the Hilbert space $\mathscr{H}$ and $f \in \mathscr{H}$, then there is a unique Fourier coefficients $\{\lambda_{\alpha}\}_{\alpha \in A}$ contained in $\mathbb{C}$ such that $f = \sum_{\alpha \in A} \lambda _{\alpha}e_{\alpha}$. Moreover, $\lambda_{\alpha} = <{f},{e_{\alpha}}>$ for $\alpha \in A$.

Proof. By Corollary above we have

$\mathscr{H} = \left\{\sum_{\alpha \in A} \lambda_{\alpha} e_{\alpha} : \lambda_{\alpha} \in \mathbb{C}, \sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 < \infty\right\}.$

Hence, $\forall f \in \mathscr{H}$, there is $\{\lambda_{\alpha}\}$, with $\sum_{\alpha \in A} |{\lambda_{\alpha}}|^2 < \infty$, such that $f = \sum_{\alpha \in A} \lambda_{\alpha}e_{\alpha}$. Then, we only need to show the uniqueness.

If $f = \sum_{\alpha \in A} \lambda_{\alpha} e_{\alpha} = \sum_{\alpha \in A}\mu_{\alpha}e_{\alpha}$. Then $0 = \sum_{\alpha \in A}(\lambda_{\alpha} - \mu_{\alpha})e_{\alpha}$.
Hence for each $\beta \in A$,

Hence, $\lambda_{\beta} = \mu_{\beta}$, $\forall \beta \in A$​.

Finally for each $\beta \in A$,

$<{f},{e_{\beta}}> = \sum_{\alpha \in A}\lambda_{\alpha}<{e_{\alpha}},{e_{\beta}}> = \lambda_{\beta}.$

Theorem. Every Hilbert space($\neq \{0\}$) possesses an orthonormal basis.

Proof. Since $\mathscr{H} \neq \{0\}$, then $\mathscr{H}$ has at least one orthonormal set. Assume, $\mathcal{F}$ denoted by the collection of all orthonormal subsets of $\mathscr{H}$. And define the ordered on $\mathcal{F}$, for $F_1, F_2 \in \mathcal{F}$, $F_1 \le F_2$ if $F_1 \subset F_2$. Then $(\mathcal{F}, \le)$ is a partial ordered set.

Let $\mathcal{A}$ is a total ordered subset of $\mathcal{F}$, let $G = \bigcup_{F \in \mathcal{A}}F$.

We prove $G$ is an orthonormal subsets of $\mathscr{H}$, hence it is an upper bound of $\mathcal{A}$

$\forall f, g \in G$, there is $F_1, F_2 \in \mathcal{A}$, such that $f \in F_1$ and $g \in F_2$. Since, $\mathcal{A}$ is total ordered, then $F_1 \bigcup F_2 \in \{F_1, F_2\} = \max\{F_1, F_2\}$. W.L.A.G $F_1 \le F_2$, hence, $f, g \in F_2$, and note that $F_2$ is an orthonormal subset, then $f \perp g$.

By Zorn’s lemma, there is a maximal element of $\mathcal{F}$, denoted by $B = \{e_{\alpha}: \alpha \in A\}$.

Next, we prove $B$ is an orthonormal basis of $\mathscr{H}$.

Let $\mathscr{K} = \overline{\textbf{Span}}B$, and if $\mathscr{K} \subsetneq \mathscr{H}$, choose $x \in \mathscr{H} \setminus \mathscr{K}$.

Let $h = x - \sum_{\alpha \in A}<{x},{e_{\alpha}}>e_{\alpha}$. Then, $h \neq 0$ and $h \notin \mathscr{K}$ (Since $<{h},{e_{\beta}}> = 0$ $\forall \beta \in A$).

Hence, let $\widetilde{G} = G\bigcup \{h/\|{h}{}\}$, then $\widetilde{G}$ is an orthonormal subsets of $\mathscr{H}$ containing $G$, which contradicts with $G$ is maximal.

Thus, $\mathscr{K} = \mathscr{H}$, $G$ is an orthonormal basis of $\mathscr{H}$. ◻

Theorem. If $\{e_{\alpha}\}_{\alpha \in A}$ and $\{f_{\beta}\}_{\beta \in B}$ are orthonormal basis for the Hilbert space $\mathscr{H}$, then $\# A = \# B$.

Proof. If $\# A < \#B$, then, there is a injective map $\tau$ from $A$ into a proper subset $C$ of $B$.

Then, $C = \{\tau(e_{\alpha})\}_{\alpha \in A}$, and $C$ is an orthonormal subset of $\mathscr{H}$. Let $\mathcal{K} = \overline{\textbf{Span} }C$. Then, $\mathcal{K} \subsetneq \mathscr{H}$, and choose $x \in \mathscr{H} \setminus \mathcal{K}$.

For each $\gamma \in A$,

$\tau(e_{\gamma}) = \sum_{\alpha \in A} <{\tau(e_{\gamma})},{e_{\alpha}}>e_{\alpha}.$

Then $\tau$ can extend to a linear map on $\mathscr{H}$. $\tau : \mathscr{H} \to \mathcal{K}$ is one-to-one and onto by define

$\tau(x) = \sum_{\gamma \in A}<{x},{e_{\gamma}}>\tau(e_{\gamma}), \quad \forall x \in \mathscr{H}.$

$\tau$ is continuous.

If the claim holds, we have $\tau^{-1}$ is continuous. Then $\overline{\textbf{Span}}A = \overline{\textbf{Span}}C$, $\mathcal{K} = \mathscr{H}$. This is contradiction!

Hence, $\#A \ge \#B$, by symmetry $\#B \ge \#A$. $\#A = \#B$.

Finally, we prove the claim

Assume $x \in \mathscr{H}$, $\|{x}\| = 1$. Then,

$\|{\tau(x)}{}\|^2 = \|{\sum_{\gamma \in A} <{x},{e_{\gamma}}> \tau(e_{\gamma})}{}\|^2 = \sum_{\gamma \in A}|{<{x},{e_{\gamma}}>}|^2 = \|{x}{}\|^2.$

Hence, $\tau$ is continuous. We complete the proof. ◻

Theorem. If $\mathscr{H}$ is a Hilbert space, then the dimension of $\mathscr{H}$, denoted $\dim \mathscr{H}$, is the cardinality of any orthonormal basis for $\mathscr{H}$.

Theorem. Two Hilbert spaces are isomorphic if and only if their dimensions are equal.

Example. $\mathbb{C}^n$ with inner product $z = (z_1, z_2, \cdots, z_n)$, $w = (w_1, w_2,\cdots, w_n) \in \mathbb{C}^n$, then

$<{z},{w}> = \sum_{k=1}^{n}z_k \overline{w_k}.$

and $\{e_k = (0,\cdots, 0, 1, 0, \cdots, 0)\}_{k = 1}^{n}$ is the orthonormal basis of $\mathbb{C}^n$. Then $\dim \mathbb{C}^n = n$.

Theorem. $\ell^2(\mathbb{Z}^+)$ with inner product $f, g \in \ell^2(\mathbb{Z}^+)$, then

$<{f},{g}> = \sum_{n = 1}^{\infty} f(n)\overline{g(n)}.$

and consider $\{e_n\}_{n \in \mathbb{Z}^+}$, where $e_n(m) = \delta_{n,m}$. Which is an orthonormal basis of $\ell^2(\mathbb{Z}^+)$. Hence, $\dim \ell^2(\mathbb{Z}^+) = \aleph_0$.

Theorem. In the Hilbert space $L^2([0,1])$, the set $\{e^{2\pi i n x}\}_{n \in \mathbb{Z}}$ is orthonormal since

$\begin{aligned} \int_{0}^{1} e^{2\pi i (m-n) x}dx =\left\{\begin{matrix} 1,& m=n\\ 0,& m\neq n \end{matrix}\right. \end{aligned}$

By Stone-Weierstrass theorem,$\overline{\textbf{Span}}^{\|{\cdot}{}\|_{\infty}}\{e^{2\pi i n x}\}_{n \in \mathbb{Z}} = C([0,1])$.
Hence,

$\overline{\textbf{Span}}^{\|{\cdot}{}\|_{2}}\{e^{2\pi i n x}\}_{n \in \mathbb{Z}} = \overline{C([0,1])}^{\|{\cdot}{}\|_2} = L^2([0,1]).$

$\{e^{2\pi i n x}\}_{n \in \mathbb{Z}}$ is an orthonormal of $L^2([0,1])$, and $\dim L^2([0,1]) = \aleph_0$.