# Traditional Predictive Analytics vs Machine Learning Methods

What is the difference between traditional predictive analytics done using statistics and its tools and, one using machine learning and deep learning? How are we leveraging machine learning and deep learning to make predictive models better? How to decide the boundary for when to use traditional models and, when to use ML and DL? We have a lot of efficient statistical tools available Stata, SPSS. They are easier to use. Also, computationally efficient.

Thanks

I am unable to find some satisfactory and convincing answer with strong points on google.

• @Dave I disagree with "to clarify that regression models are not ML", there are ML algorithms that are variants of regression, e.g. L1 regularisation was initially discovered in the neural network community before LASSO. RBF neural networks are essentially just clustering and regression. However I completely agree with "seems to be completely uninformed by statistical principles (much to its detriment IMHO)," IMHO too! ;o) Aug 27, 2022 at 12:15

This question is very hard to answer because ML is very new and not as semantically and academically defined as statistics.

However, it helps to see the problem a bit different by focusing on tools and methods as well as goals and use cases.

Per se machine learning uses statistical and mathematical algorithms to solve computational problems and specifically problems of prediction. This is true for traditional predictive analytics and tools as well.

A random forest model is not intrinsically traditional or ML and indeed I can fit it in SPSS as well as in python.

So what if any are the differences:

First, we have to understand that ML comes from a different domain than classical statistical analysis. For many cases, there will be no real difference but a CS student may call it ML while a Sociologist will call it predictive analysis.

So let's look at more specific questions of you:

1. Why not use STATA and SPSS?

You can! Indeed STATA and SPSS do have ML capabilities themselves and could be used to model modern ML algorithms. However, for many specific use cases in the ML domain, STATA and SPSS lack computational power and the ability to model large data sets efficiently.

1. Why not use traditional models like linear regression, etc. all the time?

Not every problem and every type of data fit classical methods. Linear regression, for example, cannot cope with sparse data and large instances of NAs, it cannot predict nonlinear relations and collapses really fast if we use a lot of predictors (a simple image can be converted to 786 predictors for example).

Additionally ML methods are just updating classical predictive analytics by implementing them using state of the art technology. ML is different from traditional statistics mainly in usage, where we have stuff like incremental learning and self-optimization, etc.

Also please understand that there is a huge difference between classical ML algorithms like boosted trees, random forest, Naive Bayes, etc. and deep-learning which is really, really different.

• Thanks a lot, @Fnguyen. Your answer leads me in the right direction. What pipeline should I follow when an encounter with business problems starting from statistical analysis to Machine LEarning algorithms? What limitations and constraints one should remember? Also, if you can refer to any resource, that would be great. Thanks once again Oct 21, 2019 at 8:38
• Start with explorative data analysis (EDA), if your data is big and complex (which it should be if you plan on using ML) best do that in R or python using notebook environments (Rstudio or jupyter). Depending on datasize and your machines you can model in this environment as well or switch to cloud, for advanced Ml and deep-learning if you use R you likely switch to python at this point. Finally production depends on your environment and dependencies. Oct 21, 2019 at 11:04
• "Linear regression, for example, cannot cope with sparse data", "and collapses really fast if we use a lot of predictors" I don't think that is true. Ridge regression is fine with both of those. Incremental learning for regression has been known since the 1970s and self-optimisation (e.g. ridge trace). But +1 because I agree with the basic answer. Aug 27, 2022 at 9:30

There is no difference, Machine Learning is just statistics, but using more modern techniques. The theory of statistics also applies to machine learning methods, they are no exception. Traditional statistics tends to be more interested in inference (rather than prediction) than machine learning, but there is nothing to stop you from using ML for inference as well.

A false division between statistics and machine learning, rather than viewing it as a continuum, is bad for both communities, who have a lot to learn from each other. The main reason for this is that a lot of machine learning researchers come from a computer science, physics or engineering background (engineering in my case, but I now teach CS) and simply were not aware of a lot of things that were already well known in statistics. Fortunately there are some statisticians that have become well known in ML, which helps (e.g. Vapnik). Similarly, a lot of statistics could benefit from the (on average) greater understanding of computing in the ML field.

ML tends to be more computationally demanding, but there is no real division.

• +1 Regarding your first sentence, the joke I once told when someone with a classical background in econometrics asked me what machine learning was: “It’s when you run regression from the command line.”
– Dave
Aug 27, 2022 at 11:20
• @Dave The command line is the one true user interface! ;o) Aug 27, 2022 at 12:58

It's an interesting question. According to my understanding, first of all, it will be totally incorrect to say that a statistical approach and machine learning are the same things (even though they might look like).

--Statistics plays a major role in building any machine learning model. But Machine Learning aims to predict something based on the training set and compare to a test set, whereas statistics help us build a relationship between variables.

--Take the basic example of linear regression, where it seems as if both statistical methods and ML are the same(which is not true). Linear regression is a statistical method, we can train a linear regressor and obtain the same outcome as a statistical regression model aiming to minimize the squared error between data points. But in ML, you will be dividing your dataset into a training dataset and build your model from it and test it on a 'test set' to see how it worked. But statistics just focus on the underlying relationship between the variables. Even though this can be used for predictive analysis you mentioned in the question, but the evaluation will not be on the basis of train vs test results, rather by evaluating the significance and robustness of the model parameters.

--ML can be used for building supervised and unsupervised models where statistical methods can help you forecast continuous variables, regression, and classification. ML uses it for sure but has much more objective to it than just finding a relationship.