4
$\begingroup$

What is actually the best neural network architecture for the classic MNIST digit classifying task? I couldn't find any that would claim to be the winner...

$\endgroup$
3
$\begingroup$

Actually, what I'm going to discuss is not an architecture and is like a module used in networks. It performs really well on character based data-sets like MNIST although there is other functionality for them. There is a paper called Spatial Transformer Networks written by Max Jaderberg et al. It tries to introduce an alternative for pooling layers. What it does is tries to find the canonical shape of its input by reducing transformations, like translation and rotation, or even diminishing the distortion of the inputs. It introduces a module which helps convolutional networks to really be spatial invariant. The work is amazing and the paper is easy to be read. Based on experience, using these modules may reduce the number of neurons and layers, because networks employing them won't try to learn extra stuff of the inputs, like applied transformations to the inputs. The reason is that the net will try to learn the canonical shape of inputs.

enter image description here

(a) shows the arbitrary input to the network (b) shows what spatial transformer has done and finally (c) is the output of the spatial transformer which can be used through other layers of the networks.

One of the significant achievements of this module is that it tries to enhance distorted inputs.

enter image description here

Its performance can be seen here.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes but this type of method will only work for images liike mnist $\endgroup$ – Boris Mar 25 '18 at 11:04
  • $\begingroup$ @Boris not really, It performs really good for tasks like face detection, or other object detections in the sceen. Take a look at the video. $\endgroup$ – Media Mar 25 '18 at 11:08
0
$\begingroup$

As far as I know, the state-of-the-art architecture in MNIST competitions is the 2018 RMDL (Random Multimodel Deep Learning for classification) model. Check the ArXive paper.

The model is not a Network, but an ensemble of Networks, it ranked first here. Their GitHub repository is here.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You need to take a look at the state of the art (SOTA) visualisations on Papers with Code.

The answer is "A Branching and Merging Convolutional Network with Homogeneous Filter Capsules"

| improve this answer | |
$\endgroup$
0
$\begingroup$

An ensemble of 15 CNNs classify Kaggle's MNIST digits after training on Kaggle's 42,000 images plus 25 million more images created by rotating, scaling, and shifting Kaggle's images. Learning from 25,042,000 images, this ensemble of CNNs achieves 99.75% classification accuracy. Techniques include data augmentation, nonlinear convolution layers, learnable pooling layers, ReLU activation, ensembling, bagging, decaying learning rates, dropout, batch normalization, and adam optimization. GM CDeotte explains the arch in great details here: https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist

The SOTA link shared in the other answer uses homogeneous vector capsules generated from the final filters and gives 99.85% - a full 0.1% increase. However it is not a plain vanilla Apple to Apple conversion as the Kaggle solution involves using only a subset of the MNist images provided by the hosts. Possibly using it on the full MINST database could take it closer to the SOTA score.

The real question though is - how does this compare with the accuracy of the best SOTA CNN architecture in the universe - the human mind. While there is no formal study of the same, 99.8% was the 'claimed' human benchmark and it seems to have been broken now

| improve this answer | |
New contributor
Allohvk is a new contributor to this site. Take care in asking for clarification, commenting, and answering. Check out our Code of Conduct.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.