I'm having difficulties to wrap my head around how I can prepare my dataset to train an LSTM.

Below is a screenshot of a subset representation of my dataset.

enter image description here

There are several other feature not included in this screenshot. The last column is inhospital_mortality which is 0 or 1 for each row.

Each feature was taken at a certain time x. features with the same feature_1,2,3 were taken at the same time.

My idea is that I will need to break every row (sample) as the example below: So in this case, each row would become 6 new rows.

| tc_tb1 | spo2_tb1 | g1_tb1| inhospital_mortality (label 0 | 1) |
| tc_tb2 | spo2_tb2 | g1_tb2| inhospital_mortality (label 0 | 1) |
| tc_tb3 | spo2_tb3 | g1_tb3| inhospital_mortality (label 0 | 1) |
| tc_tb6 | spo2_tb6 | g1_tb6| inhospital_mortality (label 0 | 1) |

Am I correct here? If so, how could I accomplish this dataframe manipulation in a more straight forward way? Perhaps there's a better way to reshape turn this dataset into the format I want. I wasn't able to accomplish it.

Thank you.


Welcome to the community. About your problem, you say that your last column inhospital_mortality is binary so your goal is to classify, at a certain time with some info about them, patients likely to die in hospital if I am right.

Assuming this is your goal, I think you can try a classification approach without reshaping your dataset to a time series format, since you are not interested in forecasting a sequence (or a point in that sequence), you have independent buckets of x measurements per patient, without temporal connection in your training history. You can use certain lag values (the -1, 2, 3... you already have) as informative features, and proceed with a binary classifier. Other option is, with this approach and per patient, calculate some statistical values like min, max, mean... of the 8 measurements for each type of patient indicator, while keeping the same number of rows but aggregating the attributes.

Of course you can also reshape it by having a unique date-time index with a value for each attribute type as you say, but you end up with less input features per row to predict your inhospital_mortality target value.

If you are interested in actually forecasting and eventually use an LSTM as you suggest, you might want to have a look at other answers about this type of problem like this

  • $\begingroup$ Thank you for your response. Yes, my goal is to classify if a patiend died given certain info. I agree with you that this looks like a traditional classification approach that I could use random forest or an SVM to classify. However, I've been told to use LSTM for this task to classify and I'm confused on how I can shape my data in a way that would make sense for my LSTM model. I'm not even sure if using LSTM is any useful in this case. $\endgroup$
    – bws
    Feb 26 at 14:13
  • $\begingroup$ I have read your response and it's still unclear how I could reshape my data to use in a LSTM. $\endgroup$
    – bws
    Feb 26 at 14:20
  • $\begingroup$ First, I would wonder: how many samples do I have? Does it make sense to use a train a recurrent net in you case? Second, if you cannot achieve a good enough performance with the current approach, you must set the date time as index, to have one row per time and rest of values, so it becomes a sequence $\endgroup$
    – German C M
    Feb 26 at 14:42
  • $\begingroup$ Regardless LSTM is a good approach or not for this problem (which I would say based on this info it is not), you can have a look at the answer I posted on the answer link $\endgroup$
    – German C M
    Feb 26 at 14:50
  • $\begingroup$ I see. I have about 4.5k samples. So, in my case since I don't have a feature that has a timestamp, would I have to add a new column to input timestamp? Sorry just a bit unclear on this. $\endgroup$
    – bws
    Feb 26 at 14:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.