1
$\begingroup$

I have read about bidirectional neural networks. It seems that they need input from both past and future. so lets say we are going to predict the energy use of one hour ahead having the energy use of last 168 hours as inputs to the network. So using bidirectional RNNs it also needs the future data, which is what we are going to predict! Can you please guide me about my confusion?

Thanks

$\endgroup$
0
$\begingroup$

You are correct in your general understanding of bidirectional recurrent neural networks, they do utilize information from both the past and the future.

However, they are not usually used for predicting the future. Instead they are mostly used for tasks like: Speech Recognition, Translation and Handwritten-Recognition.

For these uses, the "prediction", or more generally the output of the model, is based on a global-scale (a big chunk of text like a full sentence or a paragraph), while the bidirectional behavior works on a local-scale (single words).

In simple terms, when we want our model to predict the meaning of a full sentence. We need it to understand the meaning of the specific words composing it. But in order to do that, we use the words that come before (past) and after (future) each specific word.

$\endgroup$
3
  • $\begingroup$ Dear Mark, Thank you a lot for your explanations. however, I see in some examples like the following link that they have used it for a univariate time series prediction! Thanks very strange for me and i can not understand how it works. would you please let me know your explanation about it? please find the titme of bidirectional in the middle of this post: machinelearningmastery.com/… $\endgroup$
    – Ahmad
    Aug 23 '19 at 15:40
  • $\begingroup$ In that example a sequence 1,2,3,4,5... is split into overlapping chunks of 3 elements (1,2,3),(2,3,4)... along with the targets 4,5... A language model would generally use non-overlapping chunks (1,2,3),(4,5,6)... $\endgroup$ Jan 21 '20 at 2:40
  • $\begingroup$ So in this case the bidirectional network is 'scanning' the chunk from left to right and right to left. But it isn't looking into the future since the target is always the next value. The limitation of this method is you can to train the model with a fixed window size $\endgroup$ Jan 21 '20 at 2:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.