# Word based perplexity from char-rnn model

I'm training a character based RNN model for text prediction and want to compare it to similar models. Since most literature uses word based perplexity as a performance metric, what would be the "proper" way to calculate word based perplexity from a character based model?

## 1 Answer

Actually, there is a formula which can easily convert character based PPL and word based PPL.

$PPL = 2^{(BPC*Nc/Nw)}$

where $BPC$ is character based $PPL$, $Nc$ and $Nw$ are the number of characters and words in a test set, respectively.

The formula is not completely fair, but it at least offers a way to comparing them. The following are some reference.

[1] Hwang K, Sung W. Character-Level Language Modeling with Hierarchical Recurrent Neural Networks[J]. 2016.

[2] Graves A. Generating Sequences With Recurrent Neural Networks[J]. Computer Science, 2013.

[3] T. Mikolov, I. Sutskever, A. Deoras, H. Le, S. Kombrink, and J. Cernocky.Subword language modeling with neural networks. Technical report, Un-published Manuscript, 2012.