I'm looking for a neural network architecture, which gives pretty good performance on an image classification challenge and is still simple to implement in a framework and also reasonably fast to train. Do you have suggestions?

I'm new the neural network implementation, and in the end, I'd like to do some "research-style" playing with it.

Moreover, a pointer to an image classification data set which is not too tough for a simple network would be great. However, not something like MNIST which is so easy that almost any classifier does a good job.

PS: Which neural network framework do you recommend if I'd like to modify the way the network is trained?


1 Answer 1


I'd advice going through cs231n course, which covers this.

A simple but good architecture is using a feed forward net with one hidden layer and ReLU.

As for dataset - CIFAR-10 is good, if you don't like MNIST.

Tensorflow is quite easy to use framework.

  • $\begingroup$ But simple feedforward is not translation invariant? I'd like to work on non-centered images. Also single is unlikely to work as soon as images are not centered and I really need a hierarchy? $\endgroup$
    – Gerenuk
    Jul 29, 2017 at 19:58
  • $\begingroup$ In this case it isn't exactly a simple dataset. One of possible solutions is to center the images while processing them. If you are sure you want to use non-centered images, fnn will perform badly indeed. So you can try CNN with several layers. Something like this: conv - relu - pool - dropout - conv - relu - pool - dropout - fully connected layer - dropout - fully connected layer. $\endgroup$ Jul 30, 2017 at 2:50
  • $\begingroup$ Are you suggesting that single layer FNN on CIFAR would give OK results (also if I use grey-scale to avoid color effects)? It's not that I need non-centered, but I want to do something which doesn't diverge too much in approach from SOTA systems. If strides and pooling is for "centering" only, it would be fine. $\endgroup$
    – Gerenuk
    Jul 30, 2017 at 8:50
  • $\begingroup$ FNN with a single hidden layer can get 50%+ accuracy on CIFAR. Using grey-scale should increase the accuracy. CNN with a structure, described by me, can give 70%+ accuracy. $\endgroup$ Jul 30, 2017 at 11:00
  • 1
    $\begingroup$ I don't want to discourage you, but maybe it would be a good idea to try cs231n course to understand how all of this works? But if you wish, here are some examples (as I did in cs231n). Simple 2 layer FNN in numpy: github.com/Erlemar/cs231n_self/blob/master/assignment1/… (you can find the file with the net itself in the repo). CNN in tensorflow: github.com/Erlemar/cs231n_self/blob/master/assignment2/… $\endgroup$ Jul 30, 2017 at 15:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.