0
$\begingroup$

I wonder if that is a training strategy that if we could take a SLEEP() function at the end of each epoch.

for i in range(epochs):
    model.train()
    loss = ...
    loss.backward()
    model.eval()
    with torch.no_grad():
        # evaluation
    model.sleep()

While the model is sleeping, it may be possible to perform some operations on the model parameters. I guess maybe something like regularization or checking the continuity of the vectors. (The dreams in sleeping contain continuous pictures) Are there any papers discussing this sleep strategy?

I come up with this idea last night when I was sleeping.😴😴

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.