# How should we use a sparse matrix in classification?

I have a sparse matrix beside my data. I think these data could be beneficial but because it is sparse in usual usage is useless. Is there any method to transform it? for example, PCA, do you have any experience?

I working on titanic competition from Kaggle.com, I add nationality (base Name feature) to data. my nationality is something like: "Nordic,Scandinavian,Norway" so I have converted it to dummy variable, but then I have a sparse matrix.

I think usually sparse matrix could not be helpful for classification and we should transform it. (??? Is it true?)

I think in some way I should merge my features to reduce sparsity.

• Not sure about PCA but other algos generally accept the sparse matrix, else we need to transfer it to matrix and then reapply.. Aug 5, 2018 at 4:32
• How about using ".todense()" method in numpy to convert it to dense matrix if that is what you want. See here: stackoverflow.com/questions/30416695/… Aug 5, 2018 at 8:40
• Please add more details about the specific objective and some example data for detailed answer. To start with, you can take SVD of the sparse matrix. scikit-learn svd solver supports sparse matrix input scikit-learn.org/stable/modules/generated/… Aug 6, 2018 at 5:53

Your whole question makes sense, except for:

because it is sparse in usual usage is useless

Therefore, I will try to answer this from different angles.

# Implementation of sparse matrix

A sparse matrix is not useless because many packages contain algorithms that accept a sparse matrix as input.

You can look at Sklearn's SVMs as an example.

You need a special implementation to deal with sparse matrix because they are stored differently from a normal matrix. However, as mentioned in the comments by @Majid Mortazavi you can follow this answer to convert your sparse matrix to a normal one and use any other implementations out there. You just need to be careful with your RAM usage.

# Theoretical application of sparse matrix

A sparse matrix is not useless because many fields in machine learning make use of sparse features.

You can look at NLP as an example. A common feature used in text categorization is TF-IDF which is a sparse representation of a document.

Since the data is sparse, it is often enough to use linear models for classification, since most of the features for a data point will be $0$. This can be considered a good thing because they are easier and faster to compute and no polynomial kernels are needed.

You also mentioned PCA for dimensionality reduction but I don't understand why you would need it at this points. Seems like you are creating a sparse matrix to encode a categorical feature.

Maybe, instead of reducing your sparse matrix, you can have a look at better ways to deal with categorical features.

You can use TruncatedSVD instead of PCA for sparse matrices. The logic is similar, but it accepts sparse data such as NLP text.