I receive data from a sensor and after some preprocessing, I get data points (1-18) with 8 features in every for-loop. That is, in one for-loop I get data of size 10x8 points while in the 2nd run, I get 18x8. where [10,18] represents the number of data-points while 8 are the number of features and so on. The number of data-points differs in every for-loop run. To make it stable I padded zeros where it is less than 15 and ignore the data points if the count is more than 15. (for above two runs, data became 15x8 and 15x8)
Training Data set: I have a data set from 20 measurements file and each file has the different batch size(number of training data). Every File has a different time duration. Let's say if one cycle is 1 sec. One file ran for 384 sec while another for 500 sec.
What I did:
I am using multivariate LSTM with labels 0 and 1 to do binary classification and padding zeros to get equal sequence data points i.e, from one file (386,15,8) while from another file (1380,15,8). In the end, I am giving Keras input_shape=(15,8). The model performance is really bad, I changed many hyper-parameters. Should I avoid zero-padding and give input_shape=(None,8) or should I involve batch_size somehow? Can anyone guide me on this?
