0
$\begingroup$

I am wondering how to approach a project, where I would like to increase the number of output classes of an already trained network. I have very good reason to believe that the model has already learnt the relevant information to be able to predict this new class, that is why I would aim only for fine-tuning (also, I have a lot less data for this class than for the rest and I do not have the hardware to train from scratch).

The model that I want to use is a transformer, where the decoder's final layers are 2 fully connected layers and a layernorm. To my understanding, new classes to a network can be added by freezing the model except for the final layer(s), increase the output dimension and fine-tune only this part of the network.

Is this a reasonable approach? If yes, do you usually take the weights for these layers and just increase the size of the weight matrix with some random weights (or just try both, and see which one gives better results)?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
0
$\begingroup$

Yes, we call this transfer learning or fine tuning.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for the link. I am specifically interested in the "add new output class" part - this I feel is much less covered in articles/tutorials. $\endgroup$
    – aqua
    Commented Jun 2, 2023 at 16:31
  • 1
    $\begingroup$ Read the article carefully. It is doing exactly you said, converting an originally 1000-class model to do binary classification. $\endgroup$
    – lpounng
    Commented Jun 3, 2023 at 16:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.