Can I use Linear Regression to model a nonlinear function?

Question

I have recently started studying the basics about regression, and as a beginner I started by Linear Regression.

I read this article that says that for this particular type of regression the relationship between independent and dependent variables has to be linear, which to me implies that I can only predict "lines" with Linear regression: https://www.analyticsvidhya.com/blog/2015/08/comprehensive-guide-regression/

But then I started wondering about how to model functions like "y = log(x)" or "y= sqrt(x)" or "y=exp(x)" or "y=tan(x)" or other nonlinear functions by definition which are not "lines" but "curves".

Then I carried on doing research until I found this article that says that it is not the relationship between the independent and dependent variables that should be linear, but the final functional form passed to the model: https://medium.freecodecamp.org/learn-how-to-improve-your-linear-models-8294bfa8a731

I want to know if that is really the case, and is it always possible to do this "change" in the functional form? Also if it is possible to use linear regression for nonlinear functions, is it still correct to measure the performance of the model using R_square metric?

Thank you.

Answer 1

You are asking two different questions:

1. What is linear regression?

Linear regression means that, given a response variable $y$ and a set of predictors $x_i$, you are assuming (whether or not this is true is another matter) to model your response variable as $$ y^{(j)}=\sum_{i=1}^N x_i^{(j)}\beta^i + \epsilon^{(j)} $$ for each observation $y^{(j)}$, where $\epsilon^{(j)}$ is an error term with vanishing expectation value. The purpose of the algorithm is to find the set of $\beta^i$ to minimise the errors between the above formula and the actual values of the response.

2. May I use linear regressio to model non-linear functions?

You may use the linear regression to model anything you want, this does not necessarily mean that the results will be a good fit. The mere decision to use a model makes no assumptions on whether the underlying equation is in fact reflected by the model you choose. In case of linear regression you are essentially approximating an $N$-dimensional manifold (where all true points belong) with their projections onto a plane. Whether or not this is a good idea it depends on the data.

I want to know if that is really the case, and is it always possible to do this "change" in the functional form?

By using this or that other model you are not changing the functional form of the underlying variables. You are just dictating that the original relation (that you do not know) can be approximated by the model you choose.

Is it still correct to measure the performance of the model using R_square metric?

The $R^2$ is defined as the ratio between the residual sum of squares of your model over the residual sum of squares of the average. Basically it tells how much of the variance of the data is explained by your model compared to just taking a straight line (in correspondence of the average) passing through all your data points.