6
$\begingroup$

In particular, how much memory does a recurrent NN require as a function of the dataset size,number of nodes, etc., and how expensive is it to evaluate at runtime given a new test point?

| improve this question | | | | |
$\endgroup$
1
$\begingroup$

I've found some time ago two interesting papers about recurrent neural networks and their complexity. I guess you can use those as a reference points at least:

| improve this answer | | | | |
$\endgroup$
0
$\begingroup$

Besides the excellent references given by sebap123, from the Deep Learning Book by Ian Goodfellow et.al,

The recurrent neural network [given] is universal in the sense that any function computable by a Turing machine can be computed by such a recurrent network of a finite size. The output can be read from the RNN after a number of time steps that is asymptotically linear in the number of time steps used by the Turing machine and asymptotically linear in the length of the input .

Hope this helps!

| improve this answer | | | | |
$\endgroup$
  • $\begingroup$ I had to re-read that quote to figure out the link to the question . . . theoretical comparisons to a Turing machine are some steps away from specifics in the question, such as number of nodes vs memory use. $\endgroup$ – Neil Slater Apr 27 '17 at 12:57
  • $\begingroup$ My bad, I should have read the question details. I was thinking about asymptotic complexity. $\endgroup$ – ngub05 Apr 27 '17 at 13:08
  • $\begingroup$ Well I guess if someone searches and finds this on the title alone, then they will find your answer which could be useful to them, so perhaps leave it as is. $\endgroup$ – Neil Slater Apr 27 '17 at 13:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.