Reaching 100% accurray in Data Mining

Question

I am currently working with Topic Models, especially LDA, and now I am asking myself if it's possible to reach total accurracy regarding the results.

If I insepct the results of my Topic Model, the overall Topic assigment for each document is pretty good, but some documents got an assigned topic which does not fit very well.

So overall I got an accuraccy of approx. $85\%$.

Is it possible to reach $100\%$ accuracy in Data Mining?

Answer 1

In theory it's of course possible to reach perfect performance: if the algorithm can find what it needs in the features to correctly distinguish between classes (or clusters), then it will perform perfectly.

In reality however it's very rare that performance is perfect, because:

• Text data is noisy and extremely diverse
• Most of the time when there is a way to obtain perfect performance, there is also a simple heuristic which can do the same job more efficiently than using ML. Basically ML is used precisely because the task is hard and/or the data is complex, so it's not surprising that there are errors.

In the case of your problem, I notice that you have labels for your data but you use an unsupervised topic modeling method, right? If this is the case you might want to try using a supervised method, since the system would have more clues to find the right answer. Also you use accuracy for evaluation, so be careful: accuracy can be misleading since it doesn't give any detail about the different classes.

Answer 2

I would like to make the argument that you actually cannot have statistically speaking 100.00% accuracy even in theory but you can get really close. However, you getting too close might mean that your overfitting. This is because you cannot have statistically speaking absolute zero uncertainty in any system of more than 2 predictors that are independent or identically distributed (see footnote).

• First a counterfactual example: The uncertainty principle says that the uncertainty in physics can never be zero, its hbar/2 . The link though corrects a common misconception about this law of quantum mechanics. The uncertainty principle is not simply something in nature or reality, but of hard derived statistics about any multivariate models in physics. Thus uncertainty applies not just to practical reality but applies to any theoretical math as well. (I will let people much smarter then me introject about the meaning of uncertainty in physics.
• Going a bit broader than physics: You know text mining and information theory has one idea that I have always felt a bit was oddly named called "entropy". Now I am no text mining expert, but it seems like a lot of people liken entropy to being a type of uncertainty and not just information gained. (Personally I think they just call it uncertainty as in physics entropy and uncertainty are not the same thing)
• So is there an uncertainty principle in text mining?: The answer is yes there is in fact a limit on "entropy" or uncertainty in information theory

Thus for your data being text your accuracy is limited by the fact your entropy is limited by constant log_10(e/2) or I think is log_10(e/2/(2*pi)) = in the case of "Shannon entropy". Read the links for more detail though.

Footnote: I would dare say any single variable model or algorithm like y = m*x also cannot have 100% accuracy, but I cannot prove that mathematically (I will leave up to smart people than me to prove that case). Proving any model with y = x1 + x2 + … cannot have 100% accuracy is easy though.