What is the effect of NOT changing filter weights of a CNN during backpropagation? I changed only the fully connected layer weights while training on the MNIST dataset and still achieved almost 99 percent accuracy.
3
-
$\begingroup$ Interesting, did you start with random weights, or using weights from some previous network? Also is your accuracy measure from the training set, or from a hold out test set? $\endgroup$– Neil SlaterAug 4, 2018 at 20:12
-
$\begingroup$ @Neil Slater : I started with random gaussian weights .The accuracy measure is on test set. $\endgroup$– Abhisek DashAug 4, 2018 at 20:34
-
$\begingroup$ @Neil Slater: The accuracy almost remains the same even with different initializations of the filters. I used 2 convolution and max pool layers and an FC layer with 256 hidden neurons $\endgroup$– Abhisek DashAug 4, 2018 at 20:44
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
0
By not changing the weights of the convolutional layers of a CNN, you are essentially feeding your classifier (the fully connected layer) random features (i.e. not the optimal features for the classification task at hand).
MNIST is an easy enough image classification task that you can pretty much feed the input pixels to a classifier without any feature extraction and it will still score in the high 90s. Besides that, perhaps the pooling layers help a bit...
Try training an MLP (without the conv/pool layers) on the input image and see how it ranks. Here is an example where an MLP (1 hidden & 1 output layer) reached 98+% without any preprocessing/feature extraction.
Edit:
I'd also like to point out to another answer I wrote, which goes into more detail on why MNIST is so easy as an image classification task.
