1
$\begingroup$

When I write a network, do I have to write the whole forward property in nn.Module.forward()? I mean if I do some operations outside the net, does grad correctly flow?

For example, I have two networks, in which the output of one is the input of the other(net1 -> midresults -> net2).

If I do some operations on midresults(net1 -> midresults -> operations -> net2), can (net1+net2) be trained end to end?

$\endgroup$

1 Answer 1

1
$\begingroup$

As long as your operations are all compatible with pytorch tensors and Autograd then yes your network will be trained end-to-end.

A good rule of thumb is to ask yourself : I'm I using tensors and pytorch operators end-to-end? If yes (and unless stated otherwise in the docs), you should be pretty safe.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.