I receive data from a sensor and after some preprocessing, I get data points (1-18) with 8 features in every for-loop. That is, in one for-loop I get data of size 10x8 points while in the 2nd run, I get 18x8. where [10,18] represents the number of data-points while 8 are the number of features and so on. The number of data-points differs in every for-loop run. To make it stable I padded zeros where it is less than 15 and ignore the data points if the count is more than 15. (for above two runs, data became 15x8 and 15x8)

Training Data set: I have a data set from 20 measurements file and each file has the different batch size(number of training data). Every File has a different time duration. Let's say if one cycle is 1 sec. One file ran for 384 sec while another for 500 sec.

What I did: I am using multivariate LSTM with labels 0 and 1 to do binary classification and padding zeros to get equal sequence data points i.e, from one file (386,15,8) while from another file (1380,15,8). In the end, I am giving Keras input_shape=(15,8). The model performance is really bad, I changed many hyper-parameters. Should I avoid zero-padding and give input_shape=(None,8) or should I involve batch_size somehow? Can anyone guide me on this?

  • $\begingroup$ In order to provide insightful information, could you please provide a subset of your data and/or the implementation of your LSTM? So we can replicate your results to check possible problems. In addition, could you please develop more your first explanatory paragraph?, I can barely understand what are you stating. $\endgroup$ – Dave Oct 19 '20 at 10:54
  • 1
    $\begingroup$ Can you check now, I can't share the data but hopefully you will understand now $\endgroup$ – A R. Oct 19 '20 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.