In our discussion of vector spaces the specific nature of as a field, other than the fact that it is a field, has played
virtually no role. In this section we no longer consider vector spaces over arbitrary fields ; rather, we restrict to be the field of real or complex numbers. In the first case is called a {\it real vector space}, in the second, a complex vector space.
We all have had some experience with real vector spaces --- in fact both analytic geometry and the subject matter of vector analysis deal with these. What concepts used there can we carry over to a more abstract setting? To begin with we had in these concrete examples the idea of length; secondly we had the idea of perpendicularity, or, more generally, that of angle. These became special cases of the notion of a dot product (often called a scalar or inner product).
Let us recall some properties of dot product as it pertained to the special case of the three-dimensional real vectors. Given the vectors
where the 's and 's are real numbers, the dot
product of and , denoted by , was defined as
Note that the length of is given by and the angle between and is determined by
What formal properties does this dot product enjoy? We list a few:
(1) and if and only if ;
(2) ;
(3) ;
for any vectors and real numbers .
Everything that has been said can be carried over to complex vector spaces. However, to get geometrically reasonable definitions we must make some modifications. If we simply define for and where the 's and 's are complex numbers, then it is quite possible that with ; this is illustrated by the vector . In fact, need not even be real. If, as in the real case, we should want to represent somehow the length of , we should like that this length be real and that a nonzero vector should not have zero length.
We can achieve this much by altering the definition of dot product slightly. If denotes the complex conjugate of the complex number , returning to the and of the paragraph above let us define
For real vectors this new definition coincides with the old one; on the other hand, for arbitrary complex vectors , not only is real, it is in fact positive. Thus we have the possibility of introducing, in a natural way, a nonnegative length. However, we do lose something; for instance it is no longer true that . In fact the exact relationship between these is . Let us list a few properties of this dot product:
(1) ;
(2) , and if and only if ;
(3) ;
(4) ;
for all complex numbers and all complex vectors .
We reiterate that in what follows is either the field of real or complex numbers.
DEFINITION. The vector space over is said to be an inner product space if there is defined for any two vectors an element in such that:
(1) ;
(2) and if and only if ;
(3) ;
for any and .
A few observations about properties (1), (2), and (3) are in order. A function satisfying them is called aninner product. If is the field of complex numbers, property (1) implies that is real, and so property (2) makes sense. Using (1) and (3) we see that
We pause to look at some examples of inner product spaces.
EXAMPLE 4.4.1: In define, for and
This defines an inner product on .
EXAMPLE 4.4.2: In define for and
It is easy to verify that this defines an inner product on .
EXAMPLE 4.4.3: Let be the set of all continuous complex-valued functions on the closed unit interval . If define
We leave it to the reader to verify that this defines an inner product on .
For the remainder of this section will denote an inner product space.
DEFINITION: If then the length of
(or norm of ), written as , is defined by
.
LEMMA 4.4.1: If and then
Proof: By property (3) defining an inner product space,
But
and
Substituting these in the expression for we get the desired result.
COROLLARY: .
Proof: We have
by Lemma 4.4.1 (with ). Since and , taking square roots yields .
We digress for a moment, and prove a very elementary and familiar result about real quadratic equations.
LEMMA 4.4.2: If are real numbers such that
and for all real numbers then .
Proof: Completing the squares,
Since it is greater than or equal to for all , in particular this must be true for . Thus , and since we get .
We now proceed to an extremely important inequality, usually known as the Schwarz inequality
THEOREM 4.4.1: If then
.
Proof: If then both and , so that the result is true there.
Suppose, for the moment, that is real and . By Lemma 4.4.1, for any real number
Let , and ; for these the hypothesis of Lemma 4.4.2 is satisfied, so that . That is,
;
from this it is immediate that .
If is not real, then it certainly is not , so that is meaningful. Now,
and so it is certainly real. By the case of the Schwarz inequality discussed in the paragraph above,
since
we get
whence . Putting in that we obtain
, the desired result.
Specific cases of the Schwarz inequality are themselves of great interest. We point out two of them.
(1) If with
where and then Theorem 4.4.1 implies that
(2) If is the set of all continuous, complex-valued functions on with inner product defined by
then Theorem 4.4.1 implies that
The concept of perpendicularity is an extremely useful and important one in geometry. We introduce its analog in general inner product spaces.
DEFINITION: If then is said to be orthogonal to if .
Note that if is orthogonal to then is orthogonal to , for .
DEFINITION. If is a subspace of , the orthogonal complement of , is defined by
LEMMA 4.4.3: is a subspace of .
Proof: If then for all and all
since
Note that , for if it must be self-orthogonal, that is . The defining properties of an inner product space rule out this possibility unless .
One of our goals is to show that
.
Once this is done, the remark made above will become of some interest, for it will imply that is the direct sum of and .
DEFINITION. The set of vectors in is an orthonormal set if
(1) each is of length 1 (i.e., )
(2) for .
LEMMA 4.4.4: If is an orthonormal set, then the vectors in are linearly independent. If
then for .
Proof: Suppose that
Therefore
Since for while , this equation reduces to . Thus the 's are linearly independent.
If
then computing as above yields .
Similar in spirit and in proof to Lemma 4.4.4 is
LEMMA 4.4.5: If is an orthonormal set in and if then
is orthogonal to each of .
Proof: Computing for any , using the orthonormality of yields the result.
The construction carried out in the proof of the next theorem is one which appears and reappears in many parts of mathematics. It is a basic procedure and is known as the Gram-Schmidt orthogonalization process. Although we shall be working in a finite-dimensional inner product space, the Gram-Schmidt process works equally well in infinite-dimensional situations.
THEOREM 4.4.2: Let be a finite-dimensional inner product space; then has an orthonormal set as a basis.
Proof: Let be of dimension over and let be a basis of . From this basis we shall construct an orthonormal set of vectors; by Lemma 4.4.4 this set is linearly independent so must form a basis of .
We proceed with the construction. We seek vectors each of length 1 such that for . In fact we shall finally produce them in the following form: will be a multiple of will be in the linear span of and in the linear span of , and , and more generally, in the linear span of .
Let
then
whence . We now ask: for what value of is orthogonal to ? All we need is that
,
that is
.
Since will do the trick. Let
is orthogonal to ; since and are linearly independent, and must be linearly independent, and so . Let ; then is an orthonormal set. We continue. Let
A simple check verifies that
.
Since , and are linearly independent (for are in the linear span of and ), . Let ; then is an orthonormal set. The road ahead is now clear. Suppose that we have constructed , in the linear span of , which form an orthonormal set. How do we construct the next one, ? Merely put
That and that it is orthogonal to each of we leave to the reader. Put .
In this way, given linearly independent elements in we can construct an orthonormal set having elements. If particular, when , from any basis of we can construct an orthonormal set having elements. This provides us with the required basis for .
We illustrate the construction used in the last proof in a concrete case. Let be the real field and let be the set of polynomials, in a variable , over of degree 2 or less. In we define an inner product by: if then
Let us start with the basis of . Following the construction used,
which after the computations reduces to , and so
finally,
and so
We mentioned the next theorem earlier as one of our goals. We are now able to prove it.
THEOREM 4.4.3: If is a finite-dimensional inner product space and if is a subspace of then
.
More particularly, is the direct sum of and .
Proof: Because of the highly geometric nature of the result, and because it is so basic, we give several proofs. The first will make use of Theorem 4.4.2 and some of the earlier lemmas. The second will be motivated geometrically.
First Proof: As a subspace of the inner product space is itself an inner product space (its inner product being that of restricted to ). Thus we can find an orthonormal set in which is a basis of . If
by Lemma 4.4.5,
is orthogonal to each of and so is orthogonal to . Thus , and since
it follows that
Therefore . Since , this sum is direct.
Second Proof: In this proof we shall assume that is the field of real numbers. The proof works, in almost the same way, for the complex numbers; however, it entails a few extra details which might tend to obscure the essential ideas used.
Let ; suppose that we could find a vector such that
for all . We claim that then for all , that is, .
If , then in consequence of which
However, the right-hand side is
leading to
for all . If is any positive integer, since we have that
and so
for any positive integer . However as whence
.
Similarly, , and so
,
yielding
for all . Thus ; hence
.
To finish the second proof we must prove the existence of a such that
for all . We indicate sketchily two ways of proving the existence of such a .
Let be a basis of ; thus any is of the form
.
Let and let for . Thus
This quadratic function in the 's is nonnegative and so, by results from the calculus, has a minimum. The 's for this minimum, give us the desired vector
in .
A second way of exhibiting such a minimizing is as follows. In define a metric by
;
one shows that is a proper metric on
and is now a metric space. Let
In this metric is a compact set (prove!) and so the continuous function defined for takes on a minimum at some point . We leave it to the reader to verify that is the desired vector satisfying
for all .
COROLLARY: If is a finite-dimensional inner product space and is a subspace of then
.
Proof: If then for any , whence
.
Now
and
from these we get, since the sums are direct,
.
Since and is of the same dimension as it follows that .