3
$\begingroup$

What is actually the best neural network architecture for the classic MNIST digit classifying task? I couldn't find any that would claim to be the winner...

$\endgroup$
3
$\begingroup$

Actually, what I'm going to discuss is not an architecture and is like a module used in networks. It performs really well on character based data-sets like MNIST although there is other functionality for them. There is a paper called Spatial Transformer Networks written by Max Jaderberg et al. It tries to introduce an alternative for pooling layers. What it does is tries to find the canonical shape of its input by reducing transformations, like translation and rotation, or even diminishing the distortion of the inputs. It introduces a module which helps convolutional networks to really be spatial invariant. The work is amazing and the paper is easy to be read. Based on experience, using these modules may reduce the number of neurons and layers, because networks employing them won't try to learn extra stuff of the inputs, like applied transformations to the inputs. The reason is that the net will try to learn the canonical shape of inputs.

enter image description here

(a) shows the arbitrary input to the network (b) shows what spatial transformer has done and finally (c) is the output of the spatial transformer which can be used through other layers of the networks.

One of the significant achievements of this module is that it tries to enhance distorted inputs.

enter image description here

Its performance can be seen here.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Yes but this type of method will only work for images liike mnist $\endgroup$ – Boris Mar 25 '18 at 11:04
  • $\begingroup$ @Boris not really, It performs really good for tasks like face detection, or other object detections in the sceen. Take a look at the video. $\endgroup$ – Media Mar 25 '18 at 11:08
0
$\begingroup$

As far as I know, the state-of-the-art architecture in MNIST competitions is the 2018 RMDL (Random Multimodel Deep Learning for classification) model. Check the ArXive paper.

The model is not a Network, but an ensemble of Networks, it ranked first here. Their GitHub repository is here.

| improve this answer | |
$\endgroup$
0
$\begingroup$

You need to take a look at the state of the art (SOTA) visualisations on Papers with Code.

The answer is "A Branching and Merging Convolutional Network with Homogeneous Filter Capsules"

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.