ALA C, 2003, vecka 1, Studio 1a

 

[Kurssidan, vecka 1, vecka 2, vecka 3, vecka 4, vecka 5, vecka 6, vecka 7, Matlab]

Linjär algebra i R^n: projektion, mer om baser, Gram-Schmidt

1. Projection onto subspace V=R(A),  Ch 42.35-42.38.
Let the column vectors a,b,c in R^4 and the matrix A be:

a=[1; 2; 3; 4], b=[4; 3; 2; 1], c=2*a-3*b, A=[a b c],  B=[a b]

Note that  u=[1;1;1;0]  does not belong to V=R(A).   We will compute the projection  Pu  of  u  onto  V.   We have that  a,b, but not a,b,c, are linearly independent, so a,b is a basis for V and  V=R(A)=span(a,b,c)=R(B)=span(a,b).   Pu  belongs to V, so it is a linear combination of a,b,  i.e.,

Pu=Bx=x1a+x2b.

We will determine the coefficients x1, x2, so that the residual  u-Pu  is orthogonal to  V.  This means that

(u-Pu,v)=0  for all  v in V.

Since a,b is a basis for V, it is sufficient to take  v=a and v=b here:

(u-Pu,a)=0               or equivalently                  (Pu,a)=(u,a)
(u-Pu,b)=0                                                      (Pu,b)=(u,b)

If we insert  Pu=x1a+x2b here we get a system of two equations:

x1(a,a) + x2(b,a) = (u,a)
x1(a,b) + x2(b,b) = (u,b)

Insert the given vectors a,b,u and solve for the coefficients  x1, x2.   Hint:  the matrix of the system is
C=[a'*a  a'*b;  b'*a  b'*b].
What is the vector of the right hand side?

Form the vector v=Pu and check that  w=u-Pu is orthogonal to a, b, c, and Pu  as required.    This illustrates the proof of Theorem 42.8.

2. Change of basis, Ch 42.15, 42.44.
Let

s1=[1;2;3], s2=[3;2;1], s3=[0;1;1], S=[s1, s2, s3]
e1=[1;0;0], e2=[0;1;0], e3=[0;0;1], I=[e1, e2, e3]

Show that s1,s2,s3 are linearly independent. (Hint: solve Sx=0 by using rref(S).) Show that e1,e2,e3 are linearly independent.

e1,e2,e3 are three linearly independent vectors in R^3. Therefore they are a basis for R^3. It is called the "standard basis" or here we can call it the "old basis".

s1,s2,s3 are three linearly independent vectors in R^3. Therefore they are a basis for R^3. We call it the "new basis".

The equation x=Sy can be seen as a coordinate transformation, where x contains the components of the vector x with respect to the standard (old) basis, y contains the components of the vector x with respect to the new basis s1,s2,s3, and where S=[s1, s2, s3] contains the new basis vectors expressed with respect to the old basis. This is because x=Sy=y1s1+y2s2+y3s3 expresses x as a linear combination of s1, s2, s3 with coefficients y1, y2,  y3.

The columns of S are linearly independent, which implies that det(S) is not 0, so that the inverse matrix S-1 exists. Therefore the new components are given by y=S-1x.

Compute the new components y of

(a) x=[1;1;1]   hint: use inv(S)
(b) x=[6;4;2]
(c) x=[0;-1;-1]

Answer: (b) y=[0;2;0] because x=2*s2, (c) y=[0;0;-1] because x=-s3

3.  Orthonormal basis, Gram-Schmidt process, Ch 42.39.
Now we will re-arrange the basis s1, s2, s3 into an orthonormal basis q1, q2, q3 by means of the Gram-Schmidt process.    (Do the computations with matlab whenever possible.)

(a)  Let k1=s1, normalize q1 = k1 / || k1 || .

(b)  Let k2 = s2 - t *q1  be a linear combination of s2 and q1.  Determine the coefficient  t  so that k2 is orthogonal to q1.
(Make sure you understand the following equation.)

0 = (k2,q1) = (s2,q1) - t (q1,q1) = (s2,q1) - t,   t = ?

Normalize:  q2 = k2 / || k2 || .

Note that, with this value of t, the vector t*q1 = P1s2 is the projection of s2 onto the subspace V1=span(q1)=span(s1), and k2 = s2 - P1s2.

(c)  Let k3 = s3 - t *q1 - s*q2  be a linear combination of s3, q1, and q2.  Determine the coefficients  t, s  so that k3 is orthogonal to q1 and q2.

0 = (k3,q1) = (s3,q1) - t (q1,q1) - s (q2,q1) = (s3,q1) - t,   t =?
0 = (k3,q2) = (s3,q2) - t (q1,q2) - s (q2,q2) = (s3,q2) - s,  s =?

Normalize:  q3 = k3 / || k3 ||.

Note that, with these values of t and s, the vector t*q1 + s*q2 = P2s3 is the projection of s3 onto the subspace V2=span(q1,q2)=span(s1,s2), and k2 = s3 - P2s3.

Note that q1, q2, q3 are orthogonal to each other and that they are normalized, i.e., they are an orthonormal basis, ON basis.

(d)  Form the transformation matrix  Q = [q1 q2 q3].

4. Orthogonal matrix, Ch 42.40.
Show that Q is an orthogonal matrix by computing  Q'*Q and Q*Q'.

5. Orthogonal transformation
The equation x=Qy can be seen as a coordinate transformation, where x contains the components of the vector x with respect to the standard (old) basis, y contains the components of the vector x with respect to the new basis q1,q2,q3, and where Q=[q1, q2, q3] contains the new basis vectors expressed with respect to the old basis.

The columns of Q are orthonormal, which implies that the inverse matrix  Q-1 = QT exists. Therefore the new components are given by y=QTx.

Compute the new components y of

(a) x=[1;1;1]
(b) x=[6;4;2]
(c) x=[0;-1;-1]

In each case check that the norms of the vectors x and y are the same:  || x || = || y ||,  i.e.,  || x || = || Qx ||,  see Ch 41.37.

The formula y=QTx means that the components can be computed as  y1  =  (x,q1),  y2  =  (x,q2),  y3  =  (x,q3). Try this!  Simple, isn't it?  This is not true for a non-orthonormal basis such as s1, s2, s3,  where y=S-1x and we have to compute the inverse of S first, or solve the system of equations Sy=x.


Editor: Rickard Bergström
Last modified: Mon Jan 20 16:27:27 MET 2003