Linear regression using math or machine learning? Why even use machine learning for this?

Question

I have studied statistical math and is now taking a course in machine learning. The first example the teacher talked about is how to find a linear trend line using machine learning. Why would anybody do that? In my statistical courses we calculated linear trend-lines on datasets using far less computer power than a machine learning approach would use according to my understanding. Is linear regression using machine learning only a nice example to learn machine learning basics, or are there any real world reasons for why anyone would use machine learning to find such a simple trend line?

Answer 1

Linear Regression is Linear Regression regardless of how you calculate/estimate the parameters.

The question becomes significant in case of a large multi-variate dataset where it is not easy/fast/possible to compute the parameters using algebraic equations (aka fitting simple line). In such cases Machine Learning techniques such as Stochastic Gradient Descent (SGD) can be helpful.

Answer 2

Linear regression is the working horse in many disciplines where causal models are estimated. When you look at economics, for instance, you will find that linear regresion is widely used. The reason for this is that you can identify marginal effects (the slope of some line) very easily. However, most applications use multivariate linear regression (many $x$). Theses models/regressions are often quite elaborate, combining theoretical reasoning with statistics (called econometrics in this case). See Wooldridge for a good introduction. Have a look at "fixed effects" or "systems of equations" for instance.

When it come to predictive modeling, linear regression is also mostly used in multivariate settings, e.g. in combination with principle components (as the $x$) or as polinomial regression(s). It can be very powerful (in some settings) when it comes to prediction. Further expansion of linear regression models cover "generalised additive models" (GAM) or "shrinkage" to be used in high dimensional problems (e.g. lasso, ridge). See ISL for a very good overview.

Even if you fit only a linear trend line (single variable case), linear regression can be very helpful in many cases because you can easily calculate things like confidence bands or p-values to get a quick idea of how well your univariate line fits some data.

Answer 3

Your teacher is getting fancy with the terminology. What's going on is that you're doing the usual linear regression, which happens to be a simple, easy-to-visualize example of a wide range of models in so-called supervised learning. You are not, however, doing any kind of fancy algorithm or model just because the class is called "machine learning".