Skip to main content
added 717 characters in body
Source Link

The recurrent part of a network allows you, generally speaking, to model long and short term dependencies. So your model can have some sense of state.

This is typically advantageous if you're using timeseries. For example, if you have data from a heart rate monitor and like to like to classify between resting, stress and recovering. If your datapoint says your heart rate is at 130, it depends on whether you are recovering from high loads or something else.

Edit: I forgotten your second question.

It seems to me that recurrent just means doing a convolution, adding the result to the original input, and convolving again. This gets repeated for x number of recurrent steps. What advantage does this process actually give?

I could think off a few possible answers. By convoluting the recurrent part you kind of filtering it. So you get a cleaner signal and errors won't stack as much. Vanilla rnn suffer from exploding vanishing gradients, so this could be his approach to overcome it. Furthermore, you are embedding your features within the rcnn, which can lead, as he stated, to more paths to exploit. Which makes it less prone to overfitting, thus more generalizable.

The recurrent part of a network allows you, generally speaking, to model long and short term dependencies. So your model can have some sense of state.

This is typically advantageous if you're using timeseries. For example, if you have data from a heart rate monitor and like to like to classify between resting, stress and recovering. If your datapoint says your heart rate is at 130, it depends on whether you are recovering from high loads or something else.

The recurrent part of a network allows you, generally speaking, to model long and short term dependencies. So your model can have some sense of state.

This is typically advantageous if you're using timeseries. For example, if you have data from a heart rate monitor and like to like to classify between resting, stress and recovering. If your datapoint says your heart rate is at 130, it depends on whether you are recovering from high loads or something else.

Edit: I forgotten your second question.

It seems to me that recurrent just means doing a convolution, adding the result to the original input, and convolving again. This gets repeated for x number of recurrent steps. What advantage does this process actually give?

I could think off a few possible answers. By convoluting the recurrent part you kind of filtering it. So you get a cleaner signal and errors won't stack as much. Vanilla rnn suffer from exploding vanishing gradients, so this could be his approach to overcome it. Furthermore, you are embedding your features within the rcnn, which can lead, as he stated, to more paths to exploit. Which makes it less prone to overfitting, thus more generalizable.

Source Link

The recurrent part of a network allows you, generally speaking, to model long and short term dependencies. So your model can have some sense of state.

This is typically advantageous if you're using timeseries. For example, if you have data from a heart rate monitor and like to like to classify between resting, stress and recovering. If your datapoint says your heart rate is at 130, it depends on whether you are recovering from high loads or something else.