Inner Product Spaces

In our discussion of vector spaces the specific nature of as a field, other than the fact that it is a field, has played virtually no role. In this section we no longer consider vector spaces over arbitrary fields ; rather, we restrict to be the field of real or complex numbers. In the first case is called a {\it real vector space}, in the second, a complex vector space.

We all have had some experience with real vector spaces --- in fact both analytic geometry and the subject matter of vector analysis deal with these. What concepts used there can we carry over to a more abstract setting? To begin with we had in these concrete examples the idea of length; secondly we had the idea of perpendicularity, or, more generally, that of angle. These became special cases of the notion of a dot product (often called a scalar or inner product).

Let us recall some properties of dot product as it pertained to the special case of the three-dimensional real vectors. Given the vectors



where the 's and 's are real numbers, the dot product of and , denoted by , was defined as



Note that the length of is given by and the angle between and is determined by



What formal properties does this dot product enjoy? We list a few:

(1) and if and only if ;

(2) ;

(3) ;

for any vectors and real numbers .

Everything that has been said can be carried over to complex vector spaces. However, to get geometrically reasonable definitions we must make some modifications. If we simply define for and where the 's and 's are complex numbers, then it is quite possible that with ; this is illustrated by the vector . In fact, need not even be real. If, as in the real case, we should want to represent somehow the length of , we should like that this length be real and that a nonzero vector should not have zero length.

We can achieve this much by altering the definition of dot product slightly. If denotes the complex conjugate of the complex number , returning to the and of the paragraph above let us define



For real vectors this new definition coincides with the old one; on the other hand, for arbitrary complex vectors , not only is real, it is in fact positive. Thus we have the possibility of introducing, in a natural way, a nonnegative length. However, we do lose something; for instance it is no longer true that . In fact the exact relationship between these is . Let us list a few properties of this dot product:

(1) ;

(2) , and if and only if ;

(3) ;

(4) ;

for all complex numbers and all complex vectors .

We reiterate that in what follows is either the field of real or complex numbers.

DEFINITION. The vector space over is said to be an inner product space if there is defined for any two vectors an element in such that:

(1) ;

(2) and if and only if ;

(3) ;

for any and .

A few observations about properties (1), (2), and (3) are in order. A function satisfying them is called an inner product. If is the field of complex numbers, property (1) implies that is real, and so property (2) makes sense. Using (1) and (3) we see that





For the remainder of this section will denote an inner product space.

DEFINITION: If then the length of (or norm of ), written as , is defined by

.

LEMMA 4.4.1: If and then



COROLLARY: .


We digress for a moment, and prove a very elementary and familiar result about real quadratic equations.

LEMMA 4.4.2: If are real numbers such that and for all real numbers then .


We now proceed to an extremely important inequality, usually known as the Schwarz inequality

THEOREM 4.4.1: If then

.




The concept of perpendicularity is an extremely useful and important one in geometry. We introduce its analog in general inner product spaces.

DEFINITION: If then is said to be orthogonal to if .

Note that if is orthogonal to then is orthogonal to , for .

DEFINITION. If is a subspace of , the orthogonal complement of , is defined by



LEMMA 4.4.3: is a subspace of .


Note that , for if it must be self-orthogonal, that is . The defining properties of an inner product space rule out this possibility unless .

One of our goals is to show that

.

Once this is done, the remark made above will become of some interest, for it will imply that is the direct sum of and .

DEFINITION. The set of vectors in is an orthonormal set if

(1) each is of length 1 (i.e., )

(2) for .

LEMMA 4.4.4: If is an orthonormal set, then the vectors in are linearly independent. If



then
  for .


Similar in spirit and in proof to Lemma 4.4.4 is

LEMMA 4.4.5: If is an orthonormal set in and if then



is orthogonal to each of .


The construction carried out in the proof of the next theorem is one which appears and reappears in many parts of mathematics. It is a basic procedure and is known as the Gram-Schmidt orthogonalization process. Although we shall be working in a finite-dimensional inner product space, the Gram-Schmidt process works equally well in infinite-dimensional situations.

THEOREM 4.4.2: Let be a finite-dimensional inner product space; then has an orthonormal set as a basis.




We mentioned the next theorem earlier as one of our goals. We are now able to prove it.

THEOREM 4.4.3: If is a finite-dimensional inner product space and if is a subspace of then

.

More particularly,
is the direct sum of and .


COROLLARY: If is a finite-dimensional inner product space and is a subspace of then

.