2
$\begingroup$

i am currently working on preprocessing unstructured data (emails,logs,bug reports and irc chats).

i wish to prove preprocessing improves the content quality.

are there metrics available to prove this? that is to compare the raw data and the preprocessed data and show some quality improvement in numbers.

content quality - i mean to say that the preprocessed corpus is more suitable for mining with topic models than the raw one.

I am aware of perplexity. i dont want to use this metric as it only evaluates the topic model performance and not the corpus.Also i wish to evaluate the corpus quality even before the topic model is applied.

Can topic coherence be used as a metric to evaluate the quality of the corpus? can more coherent topics be generated from preprocessed documents than the raw one?

Improve this question
$\endgroup$
6
  • $\begingroup$ Welcome to the site. What do you mean by content quality? Emails, logs and irc chats are different in their own ways! $\endgroup$
    – Dawny33
    Feb 20, 2016 at 12:01
  • $\begingroup$ @Dawny33 content quality - i mean to say that the preprocessed corpus is more suitable for mining with topic models than the raw one. $\endgroup$
    – Hemaa mathavan
    Feb 20, 2016 at 17:10
  • $\begingroup$ i am aware of perplexity. i dont want to use this metric as it only evaluates the topic model performance and not the corpus. $\endgroup$
    – Hemaa mathavan
    Feb 20, 2016 at 17:12
  • $\begingroup$ Thanks for clarifying. I have edited your question to reflect the intent better. Let's wait for a good answer! :) $\endgroup$
    – Dawny33
    Feb 20, 2016 at 17:16
  • $\begingroup$ You asked exactly the same question here: stats.stackexchange.com/questions/197852/… $\endgroup$
    – user10169
    Feb 23, 2016 at 13:33

1 Answer 1

Reset to default
1
$\begingroup$

The corpus quality can be determined using the zipf law ,vocabulary growth curve and the pareto distribution. An interesting thesis which explains this can be found at

http://ilk.uvt.nl/downloads/pub/papers/hait/camp2008.pdf

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.