What are some methods to reduce a dataframe so I can pass it as one sample to an SVM?

Question

I need to classify participants in an NLP study into 3 classes, based on multiple sentences spoken by the participant. I performed a feature extraction on each sentence, and so I am left with a matrix of length (# of sentences spoken x feature vector length for each sentence) for each participant. So, for me, each sample is represented by a matrix of varying length, since some participants spoke more sentences than others. What are some ways for me to reduce the dimensionality of each matrix, and also standardize the length, so I can perform an SVM with each participant as a sample?

I am also interested in learning about other methods to classify my samples, if SVMs are not the best fit. Thank you.

Answer 1

SVM are not meant to solve "arbitrarily long" classification problem, therefore you have few choices:

1. use PCA for sequences, however it takes very long since it has to build a giant matrix over which it can perform PCA
2. change model, and pick a better suited one (eg RNN)
3. pad and cut your data (most often is not needed the whole phrase to predict the output)
4. introduce some prior knowledge, for example order the most recurrent words, and remove all of those that don't add anything to the meaning (be careful, this might lead to many problems, for example if you remove "not")
5. use recurrent autoencoders, to transform a sentence to a fixed size vector, over which you can perform any ML algorithm (but this might cause some problem from the POV of explainability)

In my opinion, cut&pad is the best option to start with, simple to implement and often very powerful, but this might change from context to context

Answer 2

The standard is to take the mean across each sentence, leaving you with a vector for each participant. From there you can use t-SNE, UMAP or PCA to reduce the size of each vector.