4
$\begingroup$

Someone I know has been using the Flesch-Kincaide readability score algorithm to try to gauge how well students write as a predictor of their academic success. They've been mostly unsuccessful.

While taking a UCSD Coursera course I recently learned about this algorithm and I believe that their lack of success is just a basic misunderstanding of how the algorithm works. A low score does not imply poor writing -- anything written for college level reading ability, such as U.S. law and classic literature, receives a low readability score. High readability scores imply more simplistic writing, but not that the writing is more or less correct. So it's not really measuring much in terms of proficiency.

All that is to ask this -- is there some other algorithm or metric to gauge writing quality, which would be more useful? Something that perhaps measures proper grammar and/or punctuation?

I have been and will continue to search for this myself, but if anyone has any knowledge they can share it would be greatly appreciated.

$\endgroup$
1
$\begingroup$

ARI and Coleman-Liau are basically the same thing with different numbers.

Gunning Fog and Dale-Chall are similar in that they're simple functions of how "complicated" the words and sentences are, though at least they appeal to a notion of "complicated words", though that just pushes the problem somewhere else.

There are more and more like this, and I know you're not looking for this.

It may be stating the obvious, but if I were building such a measure, I'd look at things like misspellings and grammar errors. These are fairly possible to check automatically.

Given the success of RNNs in reproducing text in the style of given input, I suspect you could quite successfully apply deep learning to learn what about a text makes it readable, or at least find whether models trained on high or low readability texts seem to find a given new input more probable.

$\endgroup$
  • $\begingroup$ Using RNNs to appraise coherence of written text was nearly my Masters dissertation thesis. The biggest obstacle was a lack of training data, which is why I abandoned the idea. $\endgroup$ – R Hill Feb 21 '17 at 11:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.