I didn't find any good explanations or papers on this topic, other than things about category based models and multigrams (parts of words). So, I came up with one myself. I'm using Java, but here's the code for the length normalization translated to Python. Also, this is assuming that you're adding the log probabilities for each word together, like KenLM does for the total sentence (or phrase) score.
# Get likelihood normalized by length and number of words.
# Since the likelihoods of each word are added together, phrases with more words tend to
# have a lower probability.
score = getLikelihood(phrase) # The normal score
numWords = phrase.count(' ')+1 # The number of words in the phrase.
# Get the absolute difference between the
# average word length of all words and the average word length of the phrase.
# Then, weight that weight by the reciprocal of the normal score, such that the
# lenWeight is less if the overall score is less.
# Intuitively, this means that we care less about normalizing by word length the more
# common the phrase is.
# This effectively solves the problem of giving a greater likelihood for long words
# that aren't even in the language model (and are given the default "<unk>" score).
lenWeight = abs(len(phrase) / numWords - avgLen) * (-1/score)
return score / (1/lenWeight)
How you want to use this normalized score, and how effective it is, depends on your application. In my case (splitting words), I used this normalized score to score each splitting option, but then also added (logarithm) the normal score of the option with context, which you could consider the prior for the option (in Naive Bayes terms). I then added an overall prior (manually picked, but could be tuned) to the original, unsplit option, for the probability of needing to split a word. The normalized score greatly helped, and with these probabilities together, I was able to get 100% accuracy for my test cases.