I tried to plot the rate of correct predictions (for the top 1 shortlist) with relation to the word's position in sentence : enter image description here

I was expecting to see a plateau sooner on the ngram setup since it needless context. However, one thing I wasn't expecting was that the prediction rate drops. In my understanding since we already have a context of 3 words, the plateau should converge asymptotically to its highest value. But both the recurrent network and the Ngram models are experiencing this drop. I have no idea why it would be.

(Note RNNLM is the name of the framework used to build the recurrent neural net, it uses 500 neurons and 100M direct connections, RNN25 is the same setup but with a training base divided by for)

Here is the sentence size distribution : enter image description here

Thanks in advance.

  • $\begingroup$ What is the x axis on your first graph? Position of word in sentence? $\endgroup$ – Neil Slater Oct 27 '15 at 13:49
  • $\begingroup$ Yes it is ! Same as the second graph $\endgroup$ – Arkantus Oct 27 '15 at 14:19
  • 3
    $\begingroup$ I think it would help if you would add a paragraph clarifying the objective and approach of your model. $\endgroup$ – Sledge Apr 24 '19 at 1:31
  • 2
    $\begingroup$ This question (even though it was upvoted a lot) could be improved: The question is unclear and the situatio/models could benefit from some elaboration. $\endgroup$ – S van Balen Aug 9 '19 at 15:17
  • $\begingroup$ Can you add error bars to get a sense of variation in the predictions? The drop in performance may be a function of noise? $\endgroup$ – Brian Spiering Aug 18 '19 at 23:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.