0
$\begingroup$

I am working on a historic time series dataset and using RNN, LSTM, GRU models, and I didn't find an answer if in test time, the h (or h, c) weights should be zeors for each batch?

If the weights should be zero, what they should be? the last updated weights from the training?

Thanks

$\endgroup$
1
$\begingroup$

The initial weights of h for GRU and h,c for LSTM are are often set to zeros, setting random weights is also an option. Also people have tried to learn the initial hidden states.

Since the hidden states are updated with every cell, if your sequences are long enough, it would not make a big difference how you initialize the hidden states.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks, In the training I initialize the weights to zero, however, in test / validation time (model.eval()), the weights also should be zero? $\endgroup$ – Yuval Asher Aug 29 at 12:02
  • $\begingroup$ @YuvalAsher, Yes. I think this is a good approach. $\endgroup$ – Akavall Aug 29 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.