Let's say that I train two neural networks on the exact same dataset.
- The first network is a
VGG19model with frozen convolutional layers so only the top dense layers are getting optimized.
- The second network is composed only of dense layers, with exactly the same architecture and hyper-parameters as the first networks fully connected layers.
Basically, I pass all my data through the convolutional layers of VGG19, saving outputs of the last convolutional layer to disk. Then, I load the data into my second model which is only composed of FC blocks and I train it to be a classifier.
Would combining the two models after training be the same as training a whole VGG19 model (with frozen conv layers) with my own FC layers?
If I understand correctly, the outputs of a trained network are entirely deterministic so in theory it should work. In practice, I have no idea if optimization algorithm takes frozen layers into account when training only the last FC layers.