Role of a known predictor in a disease prediction

Question

I am working on finding out whether the patient will develop the disease or not in a hospital.

Might be a basic info but I am just sharing it anyway.

Usually through historic data, I was able to see that patient who was in hospital or the icu for a longer time (more days) did develop the disease.

Similarly if he was in ventilation, it is also an indication of his health status

The distribution of hospital stay and ventilation hours between patients who developed disease and didn't develop disease are different and I verified them visually.

Now my question is

1) Do we need to include these as a predictor (input variable) in our model? I ask because am thinking by using these two variables, model might miss some other input variables which has an influence on outcome. For ex: I may not know whether his urea reading is an indication of whether he will develop disease or not.

Basically what I am trying to know is should we feed variables which we are confident that they will impact the outcome into the model?

Or is the model to help us know something which we aren't aware of .

Can you help me with this?

Answer 1

Basically what I am trying to know is should we feed variables which we are confident that they will impact the outcome into the model?

Or is the model to help us know something which we aren't aware of .

It depends, there's no general rule. The goal is of course to predict the target variable as well as possible, i.e. to maximize the performance. So the question is to find the right balance between:

• Giving only a few features which are known to help predict the target variables. In this case we make things as easy as possible for the learning algorithm, so that it can perform optimally with the features provided. Disadvantage: maybe there are other more subtle indications that would have helped the algorithm make better predictions.
• Feeding the learning algorithm with a very high number of features. In this case we let the algorithm find the relevant indications itself, hoping that it will catch even subtle clues that a human expert would easily miss. Disadvantage: too many features are susceptible of causing two problems: overfitting, when the model relies on very specific patterns which happened by chance in the training set; and redundancy, which can also decrease the performance with some algorithms.

The data-driven answer is to try various combinations of features on the spectrum between these two options, and evaluate the corresponding models on a validation set (preferably with cross-validation in order minimize the effect of chance). This can be done manually or using feature selection/extraction methods.

[added for clarity:] when selecting the best option between different sets of feature (or different algorithms as well), one needs to use a validation set which is distinct from the test set (and also from the training set of course), otherwise there is again a risk of overestimating the performance.