2
$\begingroup$

Could you please help me to understand it because I'm not sure if I got it correctly.

Let's say I have a dataset, of persons, with 100 features, various characteristics like height, weight, age, etc. I want to classify if are normal or abnormal. By abnormal I mean if a 20 years old man is 170cm and 150kg to identify it as abnormal.

Should I use Self Organising Map to reduce the dimensionality (these 100 features) and then K-means to classify them into normal and abnormal? Is that a correct approach? Or I can use just K-NN to classify them into normal - abnormal without any dimensionality reduction?

How many features can I use with K-NN? All the examples I've found so far use just two.

And if later I'd like to find why this person has included into the abnormal class how can I find that this happened because of these two features, his weight according to his height?

Improve this question
$\endgroup$
5
  • $\begingroup$ Is every person already marked as abnormal or normal? If not, then you can't use K-NN, because that is a supervised learning technique which requires every observation have a value for a dependent variable (like normal/abnormal, in your case). Supposing you did have every person marked as normal or abnormal, depending on how many people you have in your dataset, you very well may run into serious curse of dimensionality with 100 features, so yes, some dimensionality reduction may be helpful. $\endgroup$
    – Russell Richie
    Jan 10, 2016 at 16:46
  • $\begingroup$ So, @RussellRichie can I use SOM for dimensionality reduction and then K-Means for clustering?Is that a good approach? $\endgroup$
    – jimakos17
    Jan 10, 2016 at 17:14
  • $\begingroup$ No. it's not. Clustering is not good for classification of normal vs. anomalous. It does not know what you are looking for. It may cluster as "male" vs. "female" and that would actually be a much better clustering! Get labels, and use a decision treenfor such data. $\endgroup$
    – Has QUIT--Anony-Mousse
    Jan 10, 2016 at 21:01
  • $\begingroup$ If I get labels should I use dimensionality reduction and then decision tree or just decision tree? And would the decision tree be useful to find why I have an abnormality? Thank you @Anony-Mousse $\endgroup$
    – jimakos17
    Jan 11, 2016 at 10:42
  • $\begingroup$ As said before, a decision tree does not need dimensionality reduction. On contrary, it will likely become worse. You need labels of what is "abnormal"! $\endgroup$
    – Has QUIT--Anony-Mousse
    Jan 11, 2016 at 13:23

2 Answers 2

Reset to default
2
$\begingroup$

Should I use Self Organising Map to reduce the dimensionality?

Not the first choice. For dimensionality reduction you have more handy and basic methods like PCA and NMF (if you don't have negative values) or Archetypal Analysis which are recommended to be used first.

and then K-means to classify them into normal and abnormal?

Red alarm! Kmeans is not for classification but clustering. Be careful about the fundamental understanding of what you are about to do otherwise you get confused. If you have labels K-NN for instance is a supervised (thus proper) method.

Or I can use just K-NN to classify them into normal - abnormal without any dimensionality reduction?

Impossible. As you already know you have the problem called Curse of Dimensionality where the euclidean distances are highly distorted. You need to reduce the dimenstionality before K-NN. The only class of methods I know which may not need dimentionality reduction are Kernel Methods e.g. SVM.

How many features can I use with K-NN?

Once I examined the Curse of Dimentionality with K-Means in MATLAB and I saw after 7 dimensions everything was distorted. It of course depends on the nature of the data (distribution,etc) but this is what I got from my experiment.

And if later I'd like to find why this person has included into the abnormal class how can I find that this happened because of these two features, his weight according to his height?

This question is so fuzzy and about the goal of Machine Learning. In Machine Learning you get a set of answers (called labels) and try to learn them. You are not supposed to say if they are right or not. So if you are said a specific guy is abnormal you should ask the one who provided labels (in the terminology we call him expert) about the reason. If a guy with a certain weight and hight is called abnormal you only know this weight and hight indicate abnormal people. Why? Well, who knows?

If I did not get the last question correctly please drop me a line in comments. Would be glad to help.

My Suggestion

I recommend you to apply PCA to your data and take the first 5 PCs and feed them to K-NN or LDA. You can plot some of these PCs against each other and see how they discriminate your classes. It gives you a good impression of what is happening. You can also plot the variance or entropy of all features beforehand and drop those features whose variance or entropy is lower than a threshold.

Good Luck :)

Improve this answer
$\endgroup$
0
$\begingroup$

Without labels, I guess you are looking for Unsupervised anomaly detection technique. You may want to do some research on this topic.

Be very careful that you are not "classifying" since your data is not labeled. You are actually looking for outliers of the cluster of normal people. Your data is not labeled so any supervised technique is not applicable including K-NN. Clustering methods like K-means can be used, and you want to look for the outliers and label them as "abnormal". There are other techniques available like one class support vector machine.

With labels, it become classification problem. Any classification methods can be applied. Feature selection can be done to remove irrelevant or correlated features.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.