# Would a neural network trained on extracted features have the same accuracy as a full network with frozen layers?

Let's say that I train two neural networks on the exact same dataset.

1. The first network is a VGG19 model with frozen convolutional layers so only the top dense layers are getting optimized.
2. The second network is composed only of dense layers, with exactly the same architecture and hyper-parameters as the first networks fully connected layers.

Basically, I pass all my data through the convolutional layers of VGG19, saving outputs of the last convolutional layer to disk. Then, I load the data into my second model which is only composed of FC blocks and I train it to be a classifier.

Would combining the two models after training be the same as training a whole VGG19 model (with frozen conv layers) with my own FC layers?

If I understand correctly, the outputs of a trained network are entirely deterministic so in theory it should work. In practice, I have no idea if optimization algorithm takes frozen layers into account when training only the last FC layers.

• in theory it might work, although there are many parameters to take account of (eg finite precision effects, algorithms used, ..). But I miss the point of the question. If one of these approaches suits your goal, why baffled over such theoretical issues, that actual practice may render useless musings? – Nikos M. Mar 26 at 17:20
• I want to make sure that the results would be identical no matter which approach I choose. Otherwise I couldn't say that I used VGG19 architecture in my paper. – FloppyC0de Mar 26 at 17:30