Timeline for Why not use more than 3 hidden layers for MNIST classification?
Current License: CC BY-SA 3.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 11, 2017 at 23:33 | comment | added | S.Lim | Thank you so much for your comments. However, I wonder why the accuracy of 2-hidden-layer is better than that of 3-hidden-layer neural network. It is what I have experienced. Even if I didnt add any regularization techniques, I dont know the reasons of decreasing accuracy. I think it is not overfitting problem because training error and test error are similar. | |
| Aug 11, 2017 at 17:00 | comment | added | Neil Slater | @JanvanderVegt: Just heaping on regularisation to a deep fully-connected network will not generalise as well as CNN at image tasks, you would need a lot of data augmentation too. CNN generalisation is partially a regularisation effect (through shared weights), but the structural match to the problem is a big part of it too. | |
| Aug 11, 2017 at 15:36 | history | edited | Neil Slater | CC BY-SA 3.0 |
added 11 characters in body
|
| Aug 11, 2017 at 12:53 | comment | added | Jan van der Vegt | I would like to add that MNIST is a very 'easy' task, the data is very clean and easy to generalize on, it lies on a very low dimensional manifold which reduces the need for very deep layers. That said, I think blowing up the number of fully connected, dense layers and adding very strong regularization could get you to a better performance, close to CNNs. | |
| Aug 11, 2017 at 11:51 | history | answered | Neil Slater | CC BY-SA 3.0 |