I need to build a model which solves the following problem. I have a sequence (let's say size=n) of integers (arrivals) , which looks like this 0,0,1,5,2,...,4,8,6 , and I want to build a model which predicts the n+1 value.
What I have done so far
I treated this problem as a multiclass classification problem. Each distinct value in the sequence is represented as a class. Let's say we have 10 distinct values in the sequence. Every input is converted to a binary class matrix (i.e. for value 4 I have the matrix [0,0,0,0,1,0,0,0,0,0]) and then fed into a 2-layer LSTM.
The predictions of the model currently are really bad.
- If the input sequence is relatively big (>1000),it is dominated by zeroes , which are then fed into my model. As a result the model always predicts zero,and doesn't seem able to find the spikes into the sequence when the class is not zero.
- If it's smaller (100<n<200) , it is dominated by zeroes again,and the model can't seem to predict the occurence of the other classes.
- If it's relatively small (10<n<80) ,zeroes and the other classes are somewhat balanced but now the predictions seem to be heavily influenced by the values in the rolling window (which is relatively small as well)
What can I do to increase the accuracy of the LSTM model? Is there something I am currently doing that seems fundamentally wrong? Should I stop treating the problem as a multiclass classification?