1
$\begingroup$

I am reading through and learning how different ML methods work on different types of data, but I have faced a data set that I am not sure how ML methods, such as decision tree, Naive Bayes, and KNN, would perform on the following data sets (I'm sorry I couldn't find a clearer image). X1 ~ X6 are distinguishing attributes, while X7 ~ X14 are noise attributes. I would really appreciate how each ML method would go about fitting these data sets to the model and what their respective strengths and weaknesses are to these particular data sets.

Data A: p1

Data B:

p2

Improve this question
$\endgroup$
2
  • $\begingroup$ From the graph we can clearly see that a linear model would not work well. Therefore non linear models like Decision Tree, Random Forest, SVM will perform better. $\endgroup$
    – spectre
    Commented Oct 10, 2021 at 15:01
  • $\begingroup$ @spectre Are you talking about Data B only? $\endgroup$
    – user9532692
    Commented Oct 10, 2021 at 15:28

1 Answer 1

Reset to default
1
$\begingroup$

From the graph It is very clear that a non-linear model will perform well to distinguish between Class A and Class B . A Linear model ( Logistic Regression) give an accuracy around 50% for such datasets . A non-linear model (For example -SVM ) with a kernel trick can give you a very good accuracy . Follow this link to see the practical difference between Logistic Regression and SVM for Non-linear Dataset

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks for your explanation. However, I am particularly interested in how decision tree, Naive Bayes, and KNN will perform on those datasets. $\endgroup$
    – user9532692
    Commented Oct 10, 2021 at 16:15
  • $\begingroup$ Decision tree will perform good however Naive bayes is a weak classifier due to many assumptions it holds as a result it will not perform well on such data set and same goes for KNN . A dataset forming above graph , KNN will not be able to perform well too. $\endgroup$
    – Parth Sharma
    Commented Oct 10, 2021 at 16:20
  • $\begingroup$ As stated above a decision tree might perform well as it can model in non linear relationships as well. Naive Bayes is usually used when we have a NLP task at hand as it can perform well on large dataset. KNN might or might not perform well. $\endgroup$
    – spectre
    Commented Oct 10, 2021 at 17:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.