0
$\begingroup$

I use Pytorch exclusively to develop my model, and these are components in my model and how it works:

  • A generator
  • An encoder: a pretrained, and should not updated.
  • A loss function. Input is passed to the encoder to generate X, X is then passed to generator to generated a value Y. Y is passed to the encoder to get Z. The loss function computes error between Z and X. I detached X to prevent update the encoder. But I cannot detach Z, otherwise, the generator cannot be updated. But I also realize if Z is not detach then the encoder is also updated. In the case, how can I prevent the encoder being updated, and still allow the generator get feedback from the loss?
$\endgroup$
1
  • 2
    $\begingroup$ Hi @Jesse, welcome to the site. From your description, it is not clear how your architecture is. Can you provide more information or add a link to a reference that describes it in detail? Also, what exactly are X, Y and Z? And what does "computes error between Z and the Z and X" mean? $\endgroup$
    – noe
    Jan 10 at 7:26

1 Answer 1

0
$\begingroup$

You can use .requires_grad = False on your model components, and then train as normal.

You can look at this post for that information. https://stackoverflow.com/questions/51748138/pytorch-how-to-set-requires-grad-false

Since you only want to freeze encoder layers, is there some substring(s) in the dictionary key that can be used to identify encoder weights? If so, you can just add a conditional around the statement that sets .requires_grad = False.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.