This maybe a very broad question. Please do not delete.

However, I am curious on why do we need to learn statistics for Machine Learning?

This is given that the software libraries used already have the ‘knowledge’ encoded. Of course, a general understanding of how things work under the hood, but other than that is there a need for a more in-depth understanding?

Same question applies regarding inner workings of various models. How will a good understanding (including the maths) help?

  • 2
    $\begingroup$ because ML is statistics $\endgroup$
    – Peter
    Jun 30 '19 at 15:34
  • 1
    $\begingroup$ Some topics I've seen many people with software engineering/computer science backgrounds with little statistical background confuse: improper scoring rules, no accounting for uncertainty, difference between inference/prediction and the differences in approaches, dangers of making inferences from a predictive model and vice versa, proper validation (especially when it comes to feature selection/feature extraction), bias/variance tradeoff and why "overfitting" is context related, maximum likelihood estimation/overall mathematical statistics... $\endgroup$
    – aranglol
    Jun 30 '19 at 18:01
  • 1
    $\begingroup$ ...(which is the root of multiple ML algorithms like GLM's, GAM's, and boosting), and an overall lack of understanding of why certain algorithms work the way they do (example: the motivation for bagging in terms of variance/covariance). You are correct in that a lot of the heavy linear algebra is being done in the background, and it's not necessary to be able to prove xyz theorem off the back of your head (in my opinion). But at least having an understanding of why you are doing something, and therefore being able to identify problems goes a long way. $\endgroup$
    – aranglol
    Jun 30 '19 at 18:08

You need to learn statistics to be able to make sensible decisions about the ML models you plan to use, for example:

  • which model to use?
  • is the training-set large enough?
  • how to measure the model performance (which error measure)?
  • how to judge if the result is good enough?
  • how to (correctly!) compare different models and correct for multiple-testing?
  • how to avoid/identify selection bias?
  • how to deal with correlated features?
  • how to interpret the coefficients of the multiple linear regression model you just trained?

And most importantly: you need to learn statistics to understand that these questions (and many other ones) are important to ask.

If you want to be a software developer you may not need to learn statistics (although it will still be super useful!). But if you want to be a data-scientist, machine learning engineer or AI expert you cannot do your work properly without it.

  • $\begingroup$ Thank you very much. That clarifies. $\endgroup$
    – S Datta
    Jun 30 '19 at 16:25
  • $\begingroup$ How can statistics tell me if my training data is large enough? $\endgroup$ Jul 3 '19 at 0:26
  • 2
    $\begingroup$ @AmanMathur There is no easy answer, especially for machine learning (but data scientists may not always need ML). You may wish to read about power analysis and effect size. The sample size page on Wikipedia may be a good starting point. Here is another intersting read: How Do You Know You Have Enough Training Data? $\endgroup$
    – Louic
    Jul 3 '19 at 8:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.