Here is what I learnt recently.

Obviously, when talking about text generation RNNs we are talking about RNN language models. When asking about **word/char-based** text generation RNNs, we are asking about **word/char-based** RNN language models (LM).

**Word-based** LMs display **higher accuracy** and **lower computational cost** than **char-based** LMs. 
>[This drop of performance is **unlikely** due to the difficulty for character level model to capture longer short term memory, since also the Longer Short Term Memory (LSTM) recurrent networks work better with word-based input.][1]

This is because char-based RNN LMs require much bigger hidden layer to successfully model long-term dependencies which means higher computational costs.

Therefore, we can say that 
>[one of the **fundamental differences** between the **word level** and **character level** models is in **the number of parameters** the RNN has to access during the training and test. The smaller is the input and output layer of RNN, the larger needs to be the fully connected hidden layer, which makes the training of the model expensive.][1]

However, **char-based** RNN LMs better model languages with a rich morphology such as Finish, Turkish, Russian etc. Using **word-based** RNN LMs to model such languages is difficult if possible at all and is not advised.

The above analysis makes sense especially when you look at the output text, generated by **char-based** RNNs:

>[*The surprised in investors weren’t going to raise money. I’m not the company with the time there are all interesting quickly, don’t have to get off the same programmers.*][2]


While simple [char-based Maximum Likelihood LM][3] with a 13-character window delivers this:

>*And when she made many solid bricks. He stacked them in piles and stomped her
feet. The doctor diagnosed him with a bat. The girl and her boyfriend asked
her out.*

Of course I cherry-picked the example (actually most ML LM examples looked better than any RNN generated text I've read so far) and this tiny ML LM was trained on a simpler corpus but you get the idea: straightforward *conditional probability* generates better texts than far more complex **char-based** RNN.

**Char-based** RNN LMs can mimic grammatically correct sequences for a wide range of languages, require bigger hidden layer and computationally more expensive while **word-based** RNN LMs train faster and generate more coherent texts and yet little of the text these models generate makes actual sense. 

[1]: http://arxiv.org/abs/1511.06303
[2]: http://karpathy.github.io/2015/05/21/rnn-effectiveness/
[3]: http://nbviewer.jupyter.org/gist/yoavg/d76121dfde2618422139