I've graduated with a math major in undergrad, but mostly focused on algebra (Galois Theory, Knot theory, etc.). I work in something unrelated right now, but now I want to study machine learning. The question is, what kind of knowledge should I have if I want to really understand machine learning?

Say, here are some things I can think of, but obviously I'm missing a lot, I'm assuming.

  • "Fundamentals" (Calculus, Linear Algebra, Discrete Mathematics, Coding, etc.)
  • Probability (but what specific areas?)
  • Statistics (but what kind?)
  • Algorithms
  • Differential Equations

But what else? Or what subfields of what I've mentioned above are particularly important (i.e. Bayesian statistics)?

Edit: I am currently considering a graduate program in ML, and wanted to know if this is something I really want to do / know more about it / prepare myself.

  • $\begingroup$ This depends on what kind of work you want to do with ML, you could just use libraries and barely touch maths at all, or you could learn lots of maths and create whole new algorithms of your own. Depends on what you'd enjoy doing more. $\endgroup$ – Tasty213 Aug 22 '19 at 14:24
  • $\begingroup$ @Tasty213 I've been using ML libraries, and I wanted to learn a lot of math and really get into the nitty-gritty of ML. I understand it'll be a long process, but something I want to do. $\endgroup$ – Gust Aug 22 '19 at 14:25
  • $\begingroup$ Have you considered a postgrad in ML/AI, you get basically an entire year to study whatever your thesis is on and could really learn what you want. Before actually implementing it proving your capable to employers. $\endgroup$ – Tasty213 Aug 22 '19 at 14:27
  • $\begingroup$ read the book „Introduction to Statistical Learning“ $\endgroup$ – Peter Aug 22 '19 at 20:19

From somebody with a PhD in Probability working with AI/ML for a living. Basics of Probability theory, maybe wikipedia/cousera/... for the very basics if you never had a class in it, followed by e.g. “Probability with Martingales” by Williams. The papers here will also give you a good feel: https://en.m.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Machine_learning. As for books, this one on “classical” machine learning is pretty good and free https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/copy.html. For Deep Learning https://www.deeplearningbook.org/, this one also has the basics of probability theory. For reinforcement learning http://incompleteideas.net/book/the-book-2nd.html. For the applied side, slides from https://web.stanford.edu/class/cs224n/ and http://cs231n.stanford.edu/. As for statistics don’t bother, I’ve never seen any machine learning work reference a theorem in “pure” statistics, if there is such a thing. Of course standard undergraduate calculus and linear algebra. And learn some Python while you’re at it 😁

| improve this answer | |

I guess it would be better to start with some specialization in ml for example on coursera in oder to find interesting fields for you. There are different levels of complexity and extra knowledge isn't necessary in part of them. One more advantage is that you will have useful recommendation about materials. Then you can deepen in some area of researches and understand what you need to learn in addition. I would suggest courses by Andrew Ng.

Speaking about algorithm, "The Art of Computer Programming" by Donald Knuth is a good choice to have strong background in basiс algorithms, if you want in future to work on the development of ml methods by your self.

Bayesian methods could be an extra field of study, it's interesting area, but it doesn't mean, that you will need it in every possible ml research )

| improve this answer | |
  • $\begingroup$ Isn't the art of computer programming meant to be insanely complicated and only really necessary for programming language design? $\endgroup$ – Tasty213 Aug 22 '19 at 15:08
  • $\begingroup$ @Tasty213 you're right. This book will be good just if the author is interested in ml engineering. I've added this notice. $\endgroup$ – Lana Aug 22 '19 at 18:26

Machine learning is mostly(if not all about) function approximations and besides what you've already sad in order to do that you have to understand fields like inferential statistics, information theory(look for Kullback–Leibler divergence, mutual information), which in turn require knowledge in probability distributions(gaussian distribution) and bayesian probability(conditional probability). Also, when thinking about techniques of dimensionality reduction and clusterization, most of them are all about linear algebra(decomposition methods like eigendecomposition and singular value decomposition).

| improve this answer | |

Welcome to the site! As a professional data scientist at a Fortune 50 company, I'll try to give you some guidance. Here's what you wrote, with my helpful and highly descriptive comments in bold:

  • "Fundamentals" (Calculus, Linear Algebra, Discrete Mathematics, Coding, etc.) - WRONG!
  • Probability (but what specific areas?) - WRONG!
  • Statistics (but what kind?) - WRONG!
  • Algorithms - WRONG!
  • Differential Equations - WRONG!


Here's what my advice usually looks like to people who approach me with this question:

  • You need a lot less math than you think. Data science can now retire the Phds, no need for them - it's called Tensorflow, look into it :-)
  • You need an understanding of what is and what isn't a maching learning problem. Most people really just have "machine learning problems" that are advanced statistics. I would take a course in Applied Statistics and that will give you all you need.
  • You need to have an understanding of probability but not in the academic way. Read everything you can by Nassim Taleb. Don't think black swans can happen? I'll fire you from my team.
  • There is no "python vs R" debate . . . the answer is always python.
  • Finally, you need a ton of "soft skills". You need to be intellectually curious, you need to understand the problems people are really trying to solve, you need to present yourself well and speak clearly to any audience, not just be "the weird math guy who smells funny" in the back of the room. You need a TON of things that have absolutely nothing to do with numbers and equations.

Good luck!

P.S. I'm authorizing anyone reading this to change their SE name to "weird math guy who smells funny" - that would be an AWESOME name!

| improve this answer | |
  • 1
    $\begingroup$ he wants to learn about machine learning, not work as an engineer $\endgroup$ – BookYourLuck Aug 23 '19 at 17:56
  • $\begingroup$ @BookYourLuck This is about machine learning. You think we re-write the same math formulas over & over again? No! That's all write-once-code & Tensorflow has gone and done that. Now, how do you apply it? Can you talk to people? Can you make people believe in you & your models? Do you understand probabilities & the operational nature of algorithms? These are all critical items in the daily life of a modern data scientist. If you don't see the value in this, I don't want you on my team & I guarantee that anyone studying data science would switch places with me in a second ;) $\endgroup$ – I_Play_With_Data Aug 23 '19 at 21:28
  • 1
    $\begingroup$ Let me repeat myself. He is not asking about how to become a data scientist, he wants to learn about the mathematics underlying machine learning. He wants to know what the equations are, what they mean and how and why people invented them. He wants to learn how to come up with such equations himself. He cares about meaning, not applications. The original question was about cultural aspects of machine learning, not commercial aspects of being a data scientist in 2019. Otherwise I agree with you, except maybe you could add some listening skills to that list ;) $\endgroup$ – BookYourLuck Aug 24 '19 at 10:45

Not the answer you're looking for? Browse other questions tagged or ask your own question.