0
$\begingroup$

I want to Freeze my RoBERTa model base layer and only train on my classification layer, but i get the following error 'TFRobertaEmbeddings' object has no attribute 'parameters'. Here is my code

import transformers
model = transformers.TFRobertaModel.from_pretrained(MODEL_NAME, from_pt=True)
for param in model.roberta.embeddings.parameters():
param.requires_grad = False
Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ If I am not mistaken using the parameters method only works for pytorch layers, however the model you're using inherits from tensorflow layer. Try using the trainable attribute as described in the documentation as follows: model.roberta.embeddings.trainable = False. $\endgroup$
    – Oxbowerce
    Jan 19 at 15:58

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.