I was trying to perform text generation using only a character level feed-forward neural network after having followed this tutorial which uses LSTM. I one-hot encoded the characters of my corpus which gave a vector of length 45. Then I concatenated every 20 characters and fed this 20*45 length vector as input to an MLP with the 21st character's one hot as the output.
Thus my X (input data) shape is -> (144304, 900) and my Y (output data) shape is -> (144304, 45)
Here's the output from my code:
alice was beginning very about a grible thing was and bet she with a great come and fill feel at and beck to the darcht repeat allice waited it, put this was not an encir to the white knew the mock turtle with a sigh. ‘i only took the regular cours was be crosd it to fits some to see it was and getting she dodn as the endge of the evence, and went on to love that you were no alway not--ohe f whow to the gryphon is, who denight to goover and even stried to the dormouse, and repeated her question. ‘why did they live at the bottom of a well?’
‘tabl the without once it it howling it the duchess to herself it as eng, longing one of the door and wasting for the homend of the taits.’
‘sthing i cancus croquet with the queen to-day?’
‘i should like it very much,’ said the dryphon, ‘you first form into a line along-the sea-shore--’
‘the right!’ cried the queen nother frowing tone. any the this her for some thing is and at like the look of it at all,’ said the king: ‘however, it may kiss my hand if it likes.’
‘i’d really feeer that in a few this for some whele wish to get thing to eager to think thcapered twice, and shook note bill herself in a lell as expectant, and thowedd all have come fuconfuse it the march hare: she thought it must be the right way of speaking to a mouse: she had never done such a thing before, but she remembered having seen in her brother’s latin grammar, ‘a mouse--of a mouse--to a mouse--a mouse--o mister once to hin fent on the words with her fane with ale three king the said, and diffich a vage and so mane alice this cime.
My Question is why is MLP working similar to an RNN/LSTM. What's the advantage of using RNN/LSTM for such tasks over MLP?