Different Methods for Solving Linear Least Squares Problems 1.Introduction: In mathematics, linear least square is an approach fitting a mathematical model to data in cases where the idealized value provided by the model for any data point is expressed linearly in terms of the unknown parameters of the model. The resulting fitted model can be used to summarize the data, to predict unobserved values from the same system, and to understand the mechanisms that may underlie the system. Mathematically, linear least square is the problem of approximately solving an overdetermined system of linear equations, where the best approximation is defined as that which minimizes the sum of squared differences between the data values and their corresponding modeled values. The approach is called linear least squares since the assumed function is linear in the parameters to be estimated. 2.Methods: There are many methods for solving linear least squares problems. (1) Normal Equations Method: 【Need some explanation and a example.】 For a mn matrix A with rank n. Then the symmetric nn matrix is a positive definite . That equation has same solution as linear squares problem Ax=b. For the normal equations method, it uses some transformations: Rectangular Square Triangular (2) Orthogonal method: It needs two basic transformations. (Householder and Givens) 【Need some explanation and a example.】 (3) Singular Value Decomposition 【Need some explanation and a example.】 3. Comparison and summary: (1) For the normal equations method, it needs multiplications for forming and multiplications for solving. (2) For QR factorization, it needs multiplications for solving. Thus, if m=n both methods have similar efficiency. If m>n, the normal equations method could be more effective than QR factorization. (3) For SVD, the cost is a(), for a is the constant between 4 to 10. *For the Householder method is more accurate than the normal equation. 【need a little proof or explanation 】 *For rank-deficient or nearly rank-deficient problems, Householder with column pivoting can produce useful solution when normal equations method fails outright 【need a little proof or explanation 】 *For the SVD is more robust and reliable than Householder but more expensive. 【need a little proof or explanation 】 Reference “Linear least squares (Mathematics).” Wikipedia, Wikimedia Foundation, 9 Dec. 2017, en.wikipedia.org/wiki/Linear_least_squares_(mathematics). .

1. Different Methods for Solving Linear Least Squares Problems
1.Introduction:
In mathematics, linear least square is an approach fitting a
mathematical model to data in cases where the idealized value
provided by the model for any data point is expressed linearly
in terms of the unknown parameters of the model. The resulting
fitted model can be used to summarize the data, to predict
unobserved values from the same system, and to understand the
mechanisms that may underlie the system. Mathematically,
linear least square is the problem of approximately solving an
overdetermined system of linear equations, where the best
approximation is defined as that which minimizes the sum of
squared differences between the data values and their
corresponding modeled values. The approach is called linear
least squares since the assumed function is linear in the
parameters to be estimated.
2.Methods: There are many methods for solving linear least
squares problems.
(1) Normal Equations Method: 【Need some explanation
and a example.】
For a mn matrix A with rank n. Then the symmetric nn matrix is
a positive definite .
That equation has same solution as linear squares problem
Ax=b.
For the normal equations method, it uses some transformations:
Rectangular Square Triangular
(2) Orthogonal method: It needs two basic transformations.
(Householder and Givens) 【Need some explanation and a
example.】
(3) Singular Value Decomposition

2. 【Need some explanation and a example.】
3. Comparison and summary:
(1) For the normal equations method, it needs multiplications
for forming and multiplications for solving.
(2) For QR factorization, it needs multiplications for solving.
Thus, if m=n both methods have similar efficiency.
If m>n, the normal equations method could be more effective
than QR factorization.
(3) For SVD, the cost is a(), for a is the constant between 4 to
10.
*For the Householder method is more accurate than the normal
equation. 【need a little proof or explanation 】
*For rank-deficient or nearly rank-deficient problems,
Householder with column pivoting can produce useful solution
when normal equations method fails outright 【need a little
proof or explanation 】
*For the SVD is more robust and reliable than Householder but
more expensive. 【need a little proof or explanation 】
Reference
“Linear least squares (Mathematics).” Wikipedia, Wikimedia
Foundation, 9 Dec. 2017,
en.wikipedia.org/wiki/Linear_least_squares_(mathematics).