I have daily data on event occurences, so for each day I have a vector like [1, 0, 1] indicating that on this day event one and three occured, but event two did not occur. I want to train a model to take data from the past number of days (n_days) and to then predict the event occurences for the next day.
I believe this problems falls into the category of multi-label classification. Moreover, the data that I use has a clear time series structure. This is because events typically happen periodically (e.g. event 1 may happen every 30 days, and event 2 may happen every other day). I believe that an LSTM should be a good fit for this problem, as it can be used for multi-label classification and should also be very good for time series.
I coded a very simple LSTM to test its performance on a toy dataset, where I can create obvious patterns and feed a bunch of data, so that the LSTM should perform well. I will put it here:
model = keras.Sequential() model.add( layers.LSTM(n_events, input_shape=(x_train.shape, x_train.shape)) ) model.add(layers.Dropout(0.2)) model.add(layers.Dense(y_train.shape, activation="sigmoid")) model.compile(loss="binary_crossentropy", optimizer="adam") model.summary() model.fit(x_train, y_train, epochs=20)
In this code each entry of x_train contains 100 days from the past and each entry of y_train then contains the 101st day. I have a training set of 8000 days and a test set of 2000 days.
For events that occur every other day, or every three days, the LSTM has very good performance (for days that the event occurs it predicts about a 97% probabilty that it will occur and on days the event does not occur it predicts about 3%). My problem now is that when the amount of time between the occurences of an event increases, the LSTM performance becomes worse and worse very quickly (for 10 days for example, the predicted probability of an occurence is only 19% when the event also actually occurs).
I suspect this issue arrises because of sparsity issues, but I'm not sure if that is the case and also wouldn't be sure how to fix things if this was an issue.
How can I improve my LSTM, such that it maintains good predictions when the amount of time between two events increases?