1
$\begingroup$

On a time series problem that we try to solve using RNNs, the input usually has the shape $input features \times timesteps \times batchsize$ and we then feed this input into recurrent layers. An alternative would be to flatten the data so that the shape is $(input features \times timesteps) \times batchsize$ and use a fully connected layer for our time series task. This would clearly work and our dense network would be able to find dependencies between the data at different timesteps as well. So what is it that makes recurrent layers more powerful? I would be very thankful for an intuitive explanation.

$\endgroup$
0
$\begingroup$

The first reason is the number of parameters. The former case that you've mentioned, for each neuron there should be corresponding entries that would increase the number of training parameters. The other reason is that by employing simple feed-forward neurons you are somehow discarding the temporal information of your data which means you are discarding the sequence information in your data. This is somehow like the spatial data which is obtained by convolutional layers in CNNs.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.