2
$\begingroup$

I have a classification problem using emails as my dataset.

I would like to use scores from various readability formulas as features for the classification. However, most of them are defined for passages of text longer than 100 words, and the python library that I am using (py-readability-metrics) cannot work for text shorter than this.

Around 40% of my dataset consists of text shorter than 100 words, so if I want to use those features I can't simply drop those data points without losing a significant percentage of my dataset.

Also, in order to keep things reproducible, I wouldn't want to generalize the metrics to work for shorter texts too.

What would you recommend as an imputation value to assign as a score for those short texts?

  • Intuitively, the first that came to my mind is to find the score that means "easiest to read" for each formula, but that could not always be the case (shorter does not always mean easier to read) and also this would mess up the distributions.
  • Then I thought about the mean value, which would make more sense because I have observed that the number of words is not directly correlated to the readability score.

Is there another possible imputation value that I missed? Or is 40% missing values simply too much for these features to be worth it in your opinion?

$\endgroup$

1 Answer 1

1
$\begingroup$

I don't know much about readability scoring method, but the 100-word limit sounds quite arbitrary. I think you should analyse each scoring method available in that library and choose one which actually works well with fewer words. Then you can fork the library and edit the limit. E.g. here : https://github.com/cdimascio/py-readability-metrics/blob/3ffb97f6057ae2451599d083a69ece78a61a6fa4/readability/scorers/coleman_liau.py#L17 for the Coleman-Liau index

Then, say that after lowering the limit to 30 words, you only have 10% missing values, you can impute them using the median score.

Actually, I would optimize the limit against the classification performance metric you are using (like ROC AUC or logloss)

$\endgroup$
1
  • $\begingroup$ I am not sure that the limit of 100 is "entirely" arbitrary, since most readability scores by definition do a sampling of 100 words anyways . I see what you propose however. $\endgroup$
    – liakoyras
    Sep 19, 2022 at 11:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.