1
$\begingroup$

Morning,

I have a lot of data of which I am positive an event (target data / the event I want to predict going forward) occurred within a two week time frame, however I am unsure when it happened within this time frame.

I can get daily or more frequent feature data, but the target data only appears in 2 week gaps or in some cases 4 week gaps.

Currently I use an average of any feature during the timeframe or gather the feature data on the same day as when the target data is available but is there a more appropriate / better way?

I am using the data for machine learning purposes.

Improve this question
$\endgroup$
2
  • $\begingroup$ Do you mean the event occurs quasi-periodically every 2-4 weeks and you want to predict the next event given the history? $\endgroup$
    – Emre
    Commented Sep 30, 2016 at 6:00
  • 1
    $\begingroup$ I know if an event has occurred and the end of 2-4 weeks, but in reality it could have occurred at any point within that time period. For instance, at the end of the time frame i know if someone was ill or not, however the illness could have occurred at any point within that timeframe. However what i do get is daily updates on the inputs just not the target. $\endgroup$
    – ben121
    Commented Sep 30, 2016 at 7:19

1 Answer 1

Reset to default
1
$\begingroup$

This is referred to as "interval censoring": that is to say, you know the event occurred within an interval, but not exactly where in that interval.

I don't think there has been a lot of attention in the ML community for interval censored data (or any, really). However, there has been a reasonable amount in the statistical community, myself included. As such, I've written the R package icenReg for interval censored regression models. Regression models certainly can be used as ML tool, although these don't yet have all the bells and whistles that are more typical for ML problems (i.e. no penalized regression a la elastic net etc.).

However, icenReg at least contains a tool for generalized cross validation, although it is hidden from the public. It can be extracted by icenReg:::icenReg_cv.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks. Will look into it but sounds like a very deep subject. Is it worth bothering with or should just use averages to get the model built then refine going forward? $\endgroup$
    – ben121
    Commented Oct 4, 2016 at 7:44
  • $\begingroup$ @ben121 a good place to start would be the vignette. I don't think the programming side of things is particularly difficult; it's the same syntax as R's lm, glm etc functions. But a basic understanding of survival analysis is helpful just to be familiar with the regression models used. $\endgroup$
    – Cliff AB
    Commented Oct 4, 2016 at 13:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.