Least squares; Projections of a vector on a subspace (11/25/2008)

1. Main Points

In todays reading I learned how to make a projection of vector on a line through the origin. For thus vector there is a multiple ˆmx of x that reaches closest to y. That multiple is called the projection of y on the line spanned by x. The step from the tip of the projection ˆmx to y is the residual vector r. I will never know the value of m; but I can find ˆm which is my best estimate of m. I can evaluate the quality of our estimate by the length of the residual vector r, because IIkII is the distance between ˆmx and y, the two sides of what we had hoped would be an equality between mx and y.
One can also do curve-fitting with two parameters leads to projections onto 2-dimensional subspaces. Allowing one to find the linear combination ˆmu +ˆbv that comes closest to s. Lastly, least-squares curve-fitting with J parameters leads to projection onto subspaces spanned by J
vectors.

2. Confusing

The fundamental problem of linear modeling was hard to grasp.

3. Intruiging

The applications of such linear modeling must be almost limitless. The example of the business who wanted to investigate the return on the money that they put on advertisement and used curve-fitting with two parameters to do so is a good example of this.