Best practice to test (unit and integration) a neural network implementation

Question

I'm implementing my own neural network (with the term implementing i mean writing the code that run the neural network itself, not training).

I implemented it for didactical purpose, but i does not know how to proceede for a validation of what i have done.

So i would ask if there is some best practice for neural network test (eg : is there a dataset of neural nets with weight, input and expected output)

You can try your neural net on MNIST for sure... Also you can verify your gradient calculation — Aditya, Oct 7 at 17:54
@Aditya : sorry i have not understand your comment. Isn't MINST a dataset ? At the moment i have only the raw implementation of a network nothing more. I would like to test it before proceede with the code writing, and discover at the end of develop process that my implementation have flaw in the activation function (for example) — Skary, Oct 7 at 18:10
Please note that I've edited my answer, to add a link to this discussion which is highly pertinent, and frankly better than my existing answer. — shadowtalker, Oct 11 at 10:24

shadowtalker · Accepted Answer · 2018-10-11 10:23:30Z

Make your code testable. You should be able to write plenty of tests for individual components without having to mock up an entire training pipeline for each one. A "property-based testing" library like Hypothesis can be helpful for this.
Train a standard model on a standard dataset and make sure the accuracy numbers are similar.
See here for an in-depth discussion, with more and better advice than I've been able to provide.

1 Answer 1