1
$\begingroup$

Which classification algorithms can handle 24000 features? What are their pros and cons?

$\endgroup$
  • 3
    $\begingroup$ They all can, but be mindful of the bias-variance trade-off. Use feature selection to cull useless features, and regularization to reduce variance. $\endgroup$ – Emre Dec 28 '17 at 2:32
  • 1
    $\begingroup$ I would suggest to avoid tree-based technics which are not suited for high dimensional datasets. $\endgroup$ – Theudbald Dec 28 '17 at 17:15
2
$\begingroup$

Deep Learning Algorithms and Graphical model algorithms can handle that scale of features.

For example a typical parsing algorithm using CRF++ computes millions of features. In case of Deep Learning, A typical image of 256*256*3 has to deal with 196608 number of features where each pixel in image is a feature.

| improve this answer | |
$\endgroup$
  • $\begingroup$ I feel this answer is rather insufficient. As geompalik points out there exist many models that can "technically" handle a lot of features. In fact ,SVM are invariant to the number of features. Furthermore it's a bit absurd to consider each pixel as a feature instead of perhaps an input. $\endgroup$ – Tophat Jan 2 '18 at 13:20
  • $\begingroup$ @Tophat Why is it absurd to treat pixels as features? The inputs to say a sentiment analysis classifier are the features extracted from the sentence which serve as the input. Although no features may be extracted from an image, the pixels are none the less features. What do you think? $\endgroup$ – aneesh joshi Jan 3 '18 at 13:22
  • $\begingroup$ I think you're right. I should have really argued against just using individual pixels are the features for CRF instead of applying some neighborhood function for the pixels. $\endgroup$ – Tophat Jan 3 '18 at 13:38
2
$\begingroup$

Linear models like Logistic Regression and Support Vector Machines can also handle such feature dimensionalities. Often in text mining problems like text classification the dimensianality of the feature space equals the vocabulary size which is high.

| improve this answer | |
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.