Situations where advanced theoretical knowledge of ML helped solve a real world problem?

Question

I've invested lot of time trying to understand the theoretical aspects of Deep Learning and Neural Networks - but I'm now questioning whether it is worth it or not, given that I am someone who works mainly on the applied/business side of things.

By advanced theoretical knowledge, I mean things: Like understanding the details of how the Kernel trick works in SVM, the "no free lunch" theorem and its relationship to machine learning, the details of how neural networks are universal approximators, the VC dimension of a classifier, etc...

By applied Data Scientist, I mean someone who solves business problems using existing algorithms as opposed to someone who develops new ones.

So my question: Are there situations in an applied Data Scientist's life where such theoretical knowledge is useful? Or is this type of knowledge useful only for people who work on developing new algorithms?

Answer 1

At the end of the day, as an applied data scientist you have a bag of tools you can use to solve business problems. It's helpful to know what your tools are capable of, where each is useful, and also what their limits are. If you don't have at least some theory, knowing which method to use in which circumstance will seem like a meaningless list of details, and when things break, you may not know why - is there an error in my code, or am I making a fundamental error in applying an algorithm in a scenario where it has fundamental limitations?

You can be pragmatic in applying tools as you've seen them applied before. Theory will help you have an intuition about whether it's possible or reasonable to apply given tools to new situations that you haven't seen. It can help you answer questions like: how will this algorithm scale with the size or dimensionality of data? SVMs in particular suffer from poor scaling with data size, precisely because they rely on kernels. That kind of basic knowledge is probably more useful than being able to write down the kernel trick.