predictions based on irregular repeated measures?

Question

I need to make a model that predicts certain medical outcomes based on the answer to health-related questionnaires. Providers have patients fill out these questionnaires more than once, at irregular time intervals - for example, a patient may fill out one questionnaire at the beginning of their care episode, another 34 days later, another 12 days after that, and so on.

I have a pretty good idea of how to handle the outcomes I'm predicting; I can either use survival modeling techniques or treat it as a binary classification problem within a certain time window. However, I'm at a bit of a loss when it comes to figuring out how to incorporate the irregularly repeated input features:

• Traditional time series techniques assume observations at regular intervals.

• Recurrent neural networks usually make similar assumptions.

• By-patient deduplication or aggregation is the easiest solution, but I would rather than throw out most of my data right up front.

Does anyone know of models or strategies that are useful for this kind of problem?

Answer 1

Time-series models can reach very good results as long as there is enough data. If you don't have enough data for one person and if this data is irregular, there are good chances that no time-series models could make good predictions individually. Consequently, in order to have enough data, you can see this problem from a statistical point of view. That means considering groups of people instead of individual persons. Having groups of people would create clusters of data that could be easily studied, without needing to have 100% regular data, but regular enough for the model to recognize patterns.

Here is an example of a medication to solve heart issues having high BPM:

d0 is the start of the medical treatment for each person.

If you have irregular records, you can create groups of people with similar features, so that you can take plenty of time-based values, get their mean values, and reproduce a regular behavior.

But you must have a lot of data to reduce errors due to outliers and get relevant mean values.

Therefore, you can take similar patients (with the relevant features specific to your study, which could be weight, age, blood category, etc.), make groups, for each group record values at d+1, d+2, ..., d+N when they exist, and apply a logistic regression on all those records. If the group is big enough, you should be able to have interesting graphs and apply valid prediction models.

With this method, you could also understand by investigating why there are outliers, detect new relevant features, and improve your model.

Here are other interesting solutions for scarce and irregular data: https://www.inovex.de/de/blog/deep-learning-time-series/

https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0717-4

https://www.researchgate.net/publication/318916950_SPARTan_Scalable_PARAFAC2_for_Large_Sparse_Data