I have been working on a problem where published results using deep learning are substantially worse than results I have obtained on the same task (using the same experimental protocol) using simple statistical methods (in this case, multinomial logistic regression). I'm wondering if this is not an uncommon event. Can anybody provide concrete examples where deep learning performs demonstrably worse than simple classifier systems?
Of course there are!! And this is a great question! I myself have discovered recently what follows!
Deep learning architectures are good on unstructured data, such as time series, images, audio, graphs, etc.
Decision tree models, together with their siblings (e.g., random forests, gradient boosting trees), are still dominant on structured data.
Here is a recent paper demonstrating what I said.
Why deep learning architectures are good on unstructured data? Well, it simply boils down to the exploitation of the inductive bias, such as spatial correlations in the case of Convolutional Neural Networks (CNNs) or symmetries on signals (see, e.g., Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges).