I am building a Vanilla Neural Network in Python for my Final Year project, just using Numpy and Matplotlib, to classify the MNIST dataset. Here's the specifications of the model:

  1. One Input Layer + One Hidden Layers + One Softmax Layer
  2. Number Nodes in each layer :[784, 800, 10]
  3. Activation function used: ReLU and Softmax.
  4. Also Normalized the Train and Test set, by dividing it by 255
  5. Have used Mini Batch Gradient Descent(Mini Batch Size=4096).
  6. The Model shows very low accuracy on the Test set. Around 8% -11% accuracy.
  7. And it shows an accuracy of around 66% - 72% for the Train set.

I haven't used Regularization until now. And I don't know if that will help.

Now fed up with this, I am thinking of just implementing the model in Tensorflow and see if that works. Or is it necessary to implement it from scratch? Because I think I know how every concept works(although not being able to find the loophole in my model). What do you have to say about it?

If you want to have a look at my code here's the link. It would be great if I can get any suggestions.

P.S.: Does it create any problem if a DL model is not written using OOP?

  • $\begingroup$ What is your question exactly? How much data do you use to train and test? How long do you train for? Do you have graphs of your losses/accuracy? $\endgroup$ May 7, 2020 at 7:14
  • $\begingroup$ The problem is solved. I downloaded the test set the wrong way. $\endgroup$ May 8, 2020 at 12:18

1 Answer 1


Some quick notes:

A mini-batch size of 4096 is rather large, try training with sizes like 16, 32, or even 1 to see what happens.

Your test accuracy seems to be around random since there are 9/10 classes to predict. Does your model predict randomly (i.e. every class gets a similar score) or is it just wrong often?

is it necessary to implement it from scratch?

Necessary for who? Unless it's required for your coursework, nobody is forcing you to implement this from scratch.

P.S.: Does it create any problem if a DL model is not written using OOP?

If an implementation is correct, it's correct. Whether it's spaghetti code, whether it's a single file, whether it's beautiful, tested, and reviewed OOP software.

OOP is there to help human write more maintainable, robust, and readable software, but under the hood, the computer really couldn't care less.

  • $\begingroup$ I changed a few lines in the code and here's the result. Train set: 59269/60000 and Test set:1002/10000. Why might this happen? $\endgroup$ May 8, 2020 at 3:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.