Classification of vector sequences

Question

My dataset is comprised of vector sequences. Each vector has 50 real-valued dimensions. The number of vectors in a sequence range from 3-5 to 10-15. In other words, the length of a sequence is not fixed.

Some fair amount of the sequences (not vectors!) are annotated with a class label. My task is to learn a classifier that given a sequence of vectors, the class label for the whole sequence is computed.

I cannot tell the exact nature of the data but the nature of sequences is not temporal. Nevertheless, a vector $x_i$ cannot be interchanged with a vector $x_j$ without changing the label ($i \neq j$). In other words, the order of vectors is important. The vectors themselves are comparable, for example it makes sense to compute a dot product and use this similarity value.

My question is: what are the tools/algorithms that can help to classify such data?

UPDATE: The data has such a property that one or very few vectors influence strongly the class label.

POSSIBLE SOLUTION: After some research it looks like the Recurrent Neural Networks (RNN) fit the bill pretty naturally. The overarching idea is to pick a context size $k$, concatenate word vectors, do max pooling and feed that through classical NN. At each possible context window position in a sentence, a feature vector is built. The final feature vector is built using max pooling for example. The backpropagation is done to adjust the network's parameters. I already got some positive results (GPU is a must).

Answer 1

As you can't disclose much detail, I'm forced to be a bit generic in my answer. I hope it will be helpful nevertheless. First of all, I would only consider reducing the sequences before classification (be it by using the dot product or something else) if you can make sure that you don't lose information you need for classification afterwards. So this approach is only feasible if you have some insight into the nature of the classification. To give a simple example: if the class label is just the number of vectors in your sequence, you won't be very successfull in predicting the class label from the dot product.

Hence, I would take the full sequence as an input for classification, and impose a maximum on the sequence length you want to consider. You might do this by first finding the maximum sequence length m in your training set and then turning each sequence of 50-dimensional vectors into one vector of dimension 50*m, possibly with some missing values at the end if your sequence doesn't have maximum length. You will probably want to get rid of these missing values and you may want to simply replace them by zeros.

There's two roads you can go from here: 1.) You directly apply classification methods known to be suitable for high dimensions. Try something simple that doesn't need much tuning like naive Bayes. This way you can see whether this approach is feasible without losing too much time if it isn't. 2.) You try first to reduce the dimension and understand the nature of the classification better. You may want to use something like principal components analysis or analyse correlation/association between each vector component and the class label. If you're successful, you know how to properly reduce the dimension of your input before applying classification.

If you would like to follow any of these ideas, please keep in mind that the concrete details of your data and the classification may render any of the ideas proposed above infeasible. So please be careful to check against any details you know but can't post here before trying to make sure you're not wasting your time.

Answer 2

The data has such a property that one or very few vectors influence strongly the class label.

The best (and easiest) approach would probably be to simply train a classifer on each vector and then average the predictions across the vectors for a given sequence. The important vectors would be strongly influential in their predictions, whereas the predictions for the unimportant vectors would be close to 0.5 (or similar for a non-binary classification problem).