I am looking for a machine-learning (classification) model that can adapt to new data. By that, I don’t mean completely new samples in the online-learning sense, but rather additional details about an existing sample. Over time, for an existing sample, I learn an additional feature value that was missing before. (Once it is learned it doesn't change.) When I run an updated sample through the model, I would like to get a "more accurate" prediction.

The time and order at which new data becomes available are not consistent and features can still be missing at the end. In my training data, I only know the data that is available at the end and not when which feature value became available.

Most of the features are categorical and I am wondering if it would be enough to train a model by creating many synthetic samples where I randomly replace some of the categories with a “missing category” value. It seems to me though that this approach would not make sure that additional data would lead to a more confident prediction. The model would just see it as a new and different sample. How can I make sure that an update leads to a better prediction? Any hints are appreciated.

  • 1
    $\begingroup$ I think this' a good question but pls give more details, best by way of an example. Say the feature set available at training time for model $M_1$ is $x_1, x_2, x_3$. Are you asking what to do with model $M_1$ when feature $x_4$ appears? $\endgroup$
    – horaceT
    Feb 14 '17 at 0:29
  • $\begingroup$ Not it but the idea is the same: Consider a model that, every time a dead person is reported, predicts whether the person was murdered or died of natural causes. First, someone calls it in-> basic facts: like location, time, gender etc. Then, police arrives, files a report-> injuries etc. Then, maybe there is an autopsy etc. Throughout, the model should output an updated probability of that person being murdered. My training data consists of closed cases and all evidence recorded where usually only a few features are missing. How can I train a model such that it can also handle ongoing cases? $\endgroup$
    – oW_
    Feb 14 '17 at 1:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.