What is the normal form (equation) of linear regression? When should it be preferred to the gradient descent method?

920    Asked by ElayneBalding in Data Science , Asked on Oct 1, 2020
Answered by Elayne Balding

The normal form that is equation of linear regression is 


Here, Y=βTX is the model for the linear regression,

Y is the target or dependent variable,

β is the vector of the regression coefficient, which is arrived at using the normal equation,

X is the feature matrix containing all the features as the columns.

Note here that the first column in the X matrix consists of all 1s. This is to incorporate the offset value for the regression line.



Comparison between gradient descent and normal equation:



Here, ‘k’ is the maximum number of iterations for gradient descent, and ‘n’ is the total number of data points in the training set.

Clearly, if we have large training data, the normal equation is not preferred for use. For small values of ‘n’, the normal equation is faster than gradient descent.



Your Answer

Interviews

Parent Categories