Why is linear regression called "linear"?
Linear regression is called linear because you model your output variable (resolve it f(x)
) as a linear combination of inputs and weights (lets call them x
and w
accordingly), namely
f(x) = < w, x > + b = SUM w_i x_i + b
From a strict mathematical point of view, these models should be called affine models because of the existence of bias ( b
which should not occur in a linear transformation), but for historical reasons we use Linear instead of the name .
You can consider linear regression on top of non-linear transformations of your data, but this is just a semantic trick, as effectively it is linear regression of the transformed data, not the data itself (in data conditions, the regression set is not linear). Mathematically, it is important that linearity is preserved over. whatever you optimize, so in terms of w
. So if you believe
f(x) = <w, g(x)> + b
You can still call it linear regression (since it is linear compared to w
), but it is no longer linear regression applied to x
, but applied to g(x)
.
source to share
Posting this answer, I think it is about theta parameters. Even though the wiki link to Linear Regression starts with the following statement:
Given a dataset of n statistical units, the linear regression model assumes that the relationship between the dependent variable yi and the p-vector of the regressors xi is linear.
However, after a few lines, they mention:
Sometimes one of the regressors can be a non-linear function of another regressor or data, as in polynomial regression and segmented regression. The model remains linear if it is linear in the parameter vector Ξ².
So my guess is that linearity is not in terms of x, but in parameters.
source to share