1
$\begingroup$

I need to make a model that predicts certain medical outcomes based on the answer to health-related questionnaires. Providers have patients fill out these questionnaires more than once, at irregular time intervals - for example, a patient may fill out one questionnaire at the beginning of their care episode, another 34 days later, another 12 days after that, and so on.

I have a pretty good idea of how to handle the outcomes I'm predicting; I can either use survival modeling techniques or treat it as a binary classification problem within a certain time window. However, I'm at a bit of a loss when it comes to figuring out how to incorporate the irregularly repeated input features:

  • Traditional time series techniques assume observations at regular intervals.

  • Recurrent neural networks usually make similar assumptions.

  • By-patient deduplication or aggregation is the easiest solution, but I would rather than throw out most of my data right up front.

Does anyone know of models or strategies that are useful for this kind of problem?

$\endgroup$

1 Answer 1

1
$\begingroup$

Time-series models can reach very good results as long as there is enough data. If you don't have enough data for one person and if this data is irregular, there are good chances that no time-series models could make good predictions individually. Consequently, in order to have enough data, you can see this problem from a statistical point of view. That means considering groups of people instead of individual persons. Having groups of people would create clusters of data that could be easily studied, without needing to have 100% regular data, but regular enough for the model to recognize patterns.

Here is an example of a medication to solve heart issues having high BPM: enter image description here

d0 is the start of the medical treatment for each person.

If you have irregular records, you can create groups of people with similar features, so that you can take plenty of time-based values, get their mean values, and reproduce a regular behavior.

But you must have a lot of data to reduce errors due to outliers and get relevant mean values.

Therefore, you can take similar patients (with the relevant features specific to your study, which could be weight, age, blood category, etc.), make groups, for each group record values at d+1, d+2, ..., d+N when they exist, and apply a logistic regression on all those records. If the group is big enough, you should be able to have interesting graphs and apply valid prediction models.

With this method, you could also understand by investigating why there are outliers, detect new relevant features, and improve your model.

Here are other interesting solutions for scarce and irregular data: https://www.inovex.de/de/blog/deep-learning-time-series/

https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-018-0717-4

https://www.researchgate.net/publication/318916950_SPARTan_Scalable_PARAFAC2_for_Large_Sparse_Data

$\endgroup$
5
  • $\begingroup$ Can you clarify what this grouping accomplishes? Is the idea to construct time series that do not have missing observations, by ensuring that each group has at least one observation per time period? $\endgroup$ Sep 22, 2022 at 15:44
  • $\begingroup$ Yes: I've updated my answer with more explanations and a graph. Hope this helps. $\endgroup$ Sep 23, 2022 at 8:11
  • $\begingroup$ Thanks for the explanation. Is there a name for this technique? I'm not sure it's going to be what I want for this particular analysis but it seems like something I would want to know going forward. $\endgroup$ Sep 23, 2022 at 21:22
  • $\begingroup$ I don't know if there is a specific name. Nevertheless, I've added a link with other methods that might help. $\endgroup$ Sep 24, 2022 at 6:32
  • $\begingroup$ Does it answer your question? If not, please let me know. $\endgroup$ Oct 29, 2022 at 14:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.