Is LSTM a proper implementation for sequence classification

Question

I have 20k samples in two classes namely positive and negative. Samples are formed of 10 digits and within each sample a digit is used exactly once. For instance; 0123456789 and 9876543102 are proper samples as each digit is used once. Order of the digits are what makes samples positive or negative. A classification statement for instance, positive samples can have 2 after 5, but negatives can't. Or a more intricate one like, 90% percent of positive samples have leading 0's and none of negatives have this. For the remanining 10% percent, 512 and 315 are patterns that makes samples positive.

There is a total of 10!=3.6m possible samples. So, in order to carry out a binary classification, what should be the proper ML approach? Can LSTM's handle this? Maybe a primitive Neural Network with 10 inputs and a binary classification? Or I should look for a different ML approach? Thanks.

Answer 1

So, from what I gather, the task is to identify positive and negative sequence of digits, which abide by certain rules.

Depending on the rules themselves: for example if there are a small number of explicit and primitive rules (like each sample sequence must only have one of each digit), then might be better to hard-code the rules into a function and feed sequences through that function.

Machine learning is ideally used when you do not know the rules (function) for how the input is translated into the output.
If you do not know the rules, then an LSTM would be a good idea since it will account for the sequence of digits themselves, when classifying the sequences. The input would be the sequence of digits (possibly normalised [i.e. divide by largest digit = 9] e.g. 1 would e represented as 1/9, etc.) Then this would be a many-to-one architecture, where you feed in to the model the whole sequence of digits and then feed this output into a final softmax layer over the two classes to give you a probability distribution of the classes (i.e. how likely is this sample to be positive?)

A standard neural network would not be recommended since when you feed input into a neural network, the data is fed in parallel and does not take into account the ordering of the digits, which appears to be fundamental to your task. If you did want to use a neural network, you could potentially use positional encodings so that you can factor for ordering of the digits (https://medium.com/nlp-trend-and-review-en/positional-embeddings-7b168da36605 in the context of Natural Language, but same would apply to your problem).