2
$\begingroup$

I am using ensemble gru for my project and keeping different cell sizes for different models !For example, first gru model is of size 16 and the second is of 8 and 4 for the third model. The model is running well but I don't see any difference in the results in keeping same unit size or different. can anyone explain the impact of varying unit size for ensemble gru. ? it would be great if answer is given with theoretical proof.

$\endgroup$
  • $\begingroup$ Welcome to DS StackExchange. Please elaborate more on your question. For example: what do you mean with: "first gru model is of size 16". Do you mean GRU cells? Still, we don't know about the overall architecture of the models. Please provide more explanations on how you implemented things. As of now, it's not easy to help you. $\endgroup$ – Leevo Jan 2 at 8:43
  • $\begingroup$ @Leevo Thanks for your response.! Yes I am talking about GRU cells. I am using input layer(15,23) followed by masking and BN. since I'm using 3 layer ensemble GRU, initially with output Shape of(None, 16) for all 3 layers. To see the impact, I am varying the output for all three layers of GRU as (none,16),(none,8) and (none,4). and provide concatenated output to dense layer as input. $\endgroup$ – Mogambo0001 Jan 3 at 8:11
0
$\begingroup$

The number of units in an RNN layer determines the amount of "memory" of that layer. Higher number of units make the model (potentially) able to remember longer sequences, and to explain current values based on information that dates more back in time. If on the other side the number of units is shorter, the model at that layer won't be able to recover information way back in the past. This is tru for any Recurren architecture (simple recurrent, LSTM, GRU).

If you increase the number of units of a GRU layer and you see that there's no improvement in the quality of your model, that probably means that your time series data have "shorter memory", i.e. that you don't need to go that far back in time in order to explain current values. Of course I didn't take a look at your data, this is just the most likely reason IMHO.

| improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks for the answer. Could you please provide reference of any relevant research paper. $\endgroup$ – Mogambo0001 Jan 3 at 17:14
  • $\begingroup$ Actually, this is coming from my understanding of RNN cells. That's a great explanation from Colah's blog. Let me know if this helps, otherwise I cna find you other sources $\endgroup$ – Leevo Jan 3 at 18:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.