1
$\begingroup$

I am reading Han,Kamber,Pei's data mining book and I stumbled upon a section called "data cleaning". It tells we can use a global constant like "unknown" or "-∞" to replace missing attribute. I understand up to here. Then it says:

the mining program may mistakenly think that they form an interesting concept, since they all have value in common i.e "unknown".

What does this mean?

$\endgroup$
1

1 Answer 1

1
$\begingroup$

Any learning algorithm works by finding some patterns in the data. In unsupervised learning, this usually means finding groups of instances which share similar values for some of their features. So potentially the unknown value could be used by the algorithm as part of a pattern, and this could be a problem because it doesn't represent a real semantic information about the data.

This is what the author means, but this problem is unlikely to happen with a decent dataset: if the dataset is large enough, then the other features are likely to be different (have a lot of diverse values) in the subset of instances which have unknown for feature X. In this case the algorithm is unlikely to consider them as sharing a pattern. If it does rely on this weak pattern, then it would mean that there is no other stronger pattern in the data to rely on, so the task is unlikely to be very successful anyway.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.