3
$\begingroup$

I am doing a project related to predicting the next glucose value based on his/her past records. But in some patients, some of the recordings are missing. There are 2 scenarios of how the blood glucose values are missing in my dataset. So I'll label them as Scenario 1 and Scenario 2 in my examples.

Scenario - 1

  1. A patient has recorded the glucose values thrice a day basically before breakfast, lunch and dinner on day 30. But on day 31 he has only recorded only the breakfast and dinner. Lunch is not recorded on day 31. So how can we replace the 0 value in lunch with another value?

Scenario - 2

  1. Another patient has recorded the blood glucose values continuously from day 1 to day 40, and then he hasn't recorded the blood glucose values for another 2 days (day 41 and day 42 recordings are not there for before breakfast, before lunch and before dinner). Again he started recording the values in the day 43. So what is the best approach to tackle this kind of scenario?

I went through many articles and majority explained on replacing the mode, median or mean values for empty records. But I think mode, median is not suitable for this kind of dataset. I highly doubt whether I can use the mean also to replace values with empty records in the Scenario - 1. Can we actually use mean for replacing the empty records in Scenario - 1 or is there any other good approach?

From my knowledge I think I can't use the above three methods in replacing the values in Scenario - 2 since about 6 records are missing. If I am correct what is the best approach for the Scenario - 2.

Thank you!!!

Improve this question
$\endgroup$
2
$\begingroup$

Replacing missing values with mean, median and mode is feasible when the number of missing data is not small enough. In your cases, only a handful of data are missing. We can use the rest of the data to come up with estimates better than mean, median and mode. Let's see the possible approaches:

Scenario 1: using the reading for breakfast and lunch for day 31, try to find the nearest neighbours in day in terms of breakfast and lunch values. You can use this nearest neighbour day's dinner value as an estimate of the 31st day's dinner.

Improvement to above approach:

1) Instead of just choosing the single nearest neighbour, you may also use a kNN model.

2) If you have the dates or can figure out the day of the week (like its Sunday, Mon, Tues etc) then instead choosing the nearest neighbour among all the days, you can choose only among the same day of the week. (Nonetheless, you can figure it out by simply subtract or adding 7 to the day you want to find the values of.) Say 31st was a Sunday, then its likely that the routine would be similar to others Sundays on the data rather than a working day like Monday.

Scenario 2: You can use the values of the same days of the week to estimate the missing values. Simplest ones would be to take a median. A more powerful estimate would be by using a small neural network to estimate the next day's values given the values of the past few days (here, the number of days to consider as input would be a hyperparameter).

Improve this answer
$\endgroup$
  • $\begingroup$ Thank you very much that really helped :). I am just asking what about if I use interpolation to fill up the gaps? $\endgroup$ – HelloWorld Mar 4 '20 at 0:29
  • $\begingroup$ Which interpolation techniques do you have in mind? $\endgroup$ – user1825567 Mar 4 '20 at 12:23
  • $\begingroup$ Linear interpolation or polynomial interpolation? I just tested them by hiding some real values in the dataset. Linear interpolation is much accurate when compared with the polynomial interpolation (order = 2) but the values are not quiet similar. $\endgroup$ – HelloWorld Mar 4 '20 at 14:33
  • $\begingroup$ I think you should check the nature of your data to get better results. If the data is following some cycle or periodicity then linear interpolation won't make much sense. Although I am not an expert of the data you are dealing with but the glucose level shouldn't keep on rising, it has to be bounded. So if linear interpolation were to work, the glucose level should be increasing always as the days pass. $\endgroup$ – user1825567 Mar 4 '20 at 20:07
  • $\begingroup$ Yep, I also thought about that but the predicted values goes high and low based on the other factors...I don't know how but it goes that way :(. $\endgroup$ – HelloWorld Mar 4 '20 at 23:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.