2
$\begingroup$

i am currently working on preprocessing unstructured data (emails,logs,bug reports and irc chats).

i wish to prove preprocessing improves the content quality.

are there metrics available to prove this? that is to compare the raw data and the preprocessed data and show some quality improvement in numbers.

content quality - i mean to say that the preprocessed corpus is more suitable for mining with topic models than the raw one.

I am aware of perplexity. i dont want to use this metric as it only evaluates the topic model performance and not the corpus.Also i wish to evaluate the corpus quality even before the topic model is applied.

Can topic coherence be used as a metric to evaluate the quality of the corpus? can more coherent topics be generated from preprocessed documents than the raw one?

$\endgroup$
6
  • $\begingroup$ Welcome to the site. What do you mean by content quality? Emails, logs and irc chats are different in their own ways! $\endgroup$
    – Dawny33
    Feb 20 '16 at 12:01
  • $\begingroup$ @Dawny33 content quality - i mean to say that the preprocessed corpus is more suitable for mining with topic models than the raw one. $\endgroup$ Feb 20 '16 at 17:10
  • $\begingroup$ i am aware of perplexity. i dont want to use this metric as it only evaluates the topic model performance and not the corpus. $\endgroup$ Feb 20 '16 at 17:12
  • $\begingroup$ Thanks for clarifying. I have edited your question to reflect the intent better. Let's wait for a good answer! :) $\endgroup$
    – Dawny33
    Feb 20 '16 at 17:16
  • $\begingroup$ You asked exactly the same question here: stats.stackexchange.com/questions/197852/… $\endgroup$
    – user10169
    Feb 23 '16 at 13:33
1
$\begingroup$

The corpus quality can be determined using the zipf law ,vocabulary growth curve and the pareto distribution. An interesting thesis which explains this can be found at

http://ilk.uvt.nl/downloads/pub/papers/hait/camp2008.pdf

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.