Scikit MLPClassifier vs. Tensorflow DNNClassifier

Question

I am trying to understand the differences between Scikit MLPClassifier and Tensorflow DNNClassifier for classification task and hoping that some experts can share a light. as far as I understand, both supports backpropagation, activation functions (inc. relu), optmizer (sgd/adam). MLPClassifier can also have deep neural networks by specifying the # of hidden layers and nodes. The only difference between two that I can see is DNNClassifier supports GPU training while MLPClassifer does not. Outside of GPU support, are there other differences between them? Why do one want to use Tensorflow's DNNClassifier for basic deep neural network training (I am talking basic feed-forard NN here not the CNN, RNN, LSTM, etc.) than Scikit's MLPClassifier?

Thanks!

Answer 1

Maybe you want to use a more complicated activation function - i.e. leakyReLU or you want to add batch normalization. From a brief look at the API there is no option for batchnorm in SKlearn. Also it appears that you can't use dropout with MLPClassifier.

You might also feel more comfortable with tensorflow features - i.e. tensorboard, or you might feel more comfortable with SKlearn. To each his own.

Answer 2

MLPClassifier and DNNClassifier are both implementations of the simplest feed-forward neural network. So in principle, they are the same.

Tensorflow is a deep learning library. scikit-learn is a more traditional machine learning library.

scikit-learn have very limited coverage for deep learning, only MLPClassifier and MLPregressor, which are the basic of basics. The devs of scikit-learn focus on a more traditional area of machine learning and made a deliberate choice to not expand too much into the deep learning area.

Tensorflow, on the other hand, is dedicated to deep learning. You can compose much very complex deep learning model with it. And GPU support is a big big deal in deep learning, we really need that 1000x speed up to get any meaningful work done in deep learning.

You can play with toy data using MLPClassifier but nothing more. That being said, the scikit-learn source code and documentation are so accessible. It is amazing for learning. I highly recommend you work through the MLPClassifier source code if you just started learning deep learning.