How to classify a dataset containing variable size list of lists?

Question

I have a dataset which has a list of lists as an input (each row) and the labels are in order of (0-9). The inside lists are of two lengths, 8 and 10. Each input list is of variable length approximately 100. And the order of the all the lists are important.

The inputs look like this (Example of only first row):

[[21.0, 24.0, 144.0, 31.0, 23.0, 21.0, 23.0, 24.0], [0.96099853515625, -0.2310791015625, 0.138427734375,-0.06182861328125, -0.2490234375, -0.478515625, 0.87890625, -1.5, -2.5625, 0.0], [20.0, 23.0, 125.0, 34.0, 23.0, 20.0, 24.0, 22.0], [0.96099853515625, -0.2314453125, 0.13818359375, -0.061767578125, -0.2490234375, -0.48583984375, 0.8740234375, -1.375, -2.0625, -0.375], ... , [20.0, 24.0, 130.0, 36.0, 23.0, 23.0, 37.0, 19.0], [0.96099853515625, -0.2315673828125, 0.137939453125, -0.06182861328125, -0.244140625, -0.48583984375, 0.8916015625, -0.0625, -1.0625, -0.5]]

Please guide me how can I process this data and finally classify.

Answer 1

There are (at least) two broad approaches that you might consider.

First, you could try to summarize each variable length row of your data into a fixed length row. In your case you might include a count of how many 8-element sublists it had, a count of how many 10-element sublists it had, the average of all the 8-element lists, the standard deviation of the means of each 8-element list and so forth. This would give you a simple representation of your data that would easily fit into standard algorithms.

Alternatively, you could try a generative modelling approach where you specify what the joint probability distribution of your dataset and labels is and attempt to estimate that distribution. You can then predict a label by conditioning the distribution on a particular row of your dataset.