2
$\begingroup$

I would like to ask what is the goal of fine-tuning a VGGNet on my dataset. What does fine-tuning mean? Does it mean to change the weights or keep the weight values?

$\endgroup$
6
$\begingroup$

Fine tuning means changing the weights such that the VGGNet can perform the task you want in your dataset. The reason why fine-tuning is not called training (which is what you are doing) is because it implies that you already use a network that has been trained on a dataset. However, the concept is the same as training, but you just happen to it do with a convenient set of initial weights.

| improve this answer | |
$\endgroup$
  • $\begingroup$ What do you mean by " but you just happen to it do with a convenient set of initial weights."? $\endgroup$ – N.IT Apr 28 '18 at 10:53
  • 1
    $\begingroup$ The initial weights are the result on training the network in another set of images. This is a better initialization than choosing random weights. $\endgroup$ – David Masip Apr 28 '18 at 11:13
2
$\begingroup$

Basically fine-tuning or transfer learning is used for situations where you don't have so much data or computation time or maybe computation capacity to train a whole network from scratch. In this way, you usually replace the last layers of a pre-trained network with new ones. The first layers have already found the needed good features and last layers try to classify, you replace them because you have your own labels. Take a look at here for considering how it should be applied and here for more details. You can keep weights, also called freezing, or let the gradient-based algorithms change them, but usually freezing first layers and training last layers is faster.

| improve this answer | |
$\endgroup$
2
$\begingroup$

Networks like VGGNet have huge numbers of parameters (see Appendix D for details, but it's something like 135 million). Training such a big network takes a lot of data and a lot of time. There is ImageNet, which has 1000 classes and many thousand images. For ImageNet, some people already trained VGGNet and provided the parameters.

One way of finetuning is to replace only the last layer (about 4 million parameters) and add a new layer which has the number of classes you need. Then you can freeze all other weights and train on your data. Hence you will have lower training time and you will likely get better results (at least on similar datasets ... for datasets which look very different, there seems not to be documented conclusive evidence)

| improve this answer | |
$\endgroup$
  • $\begingroup$ Professor Andrew Ng has investigated what I've said in details in the third course of his DL specialization. $\endgroup$ – Media Apr 28 '18 at 14:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.