I am creating a model that gathers data from multiple sources and determines a confidence level for an instance that is common across those sources (ie. all sources have different features, but all have a row for 'Foo', 'Bar', etc that I want a confidence level for). The issue is that I will not always have data from all the sources, affecting the number of features I have when running the final trained model.

I have thought about creating a different model for each permutation of data inputs that I have ie. if there are possible inputs from data sources A, B, and C then I would have a model trained for (A, B, C), (A, B), (A, C), and (B, C), but this can quickly become a lot of models.

I also thought to make the features of the data that are not present 0, indicating to the model there should be a lower confidence, but the represented data might actually have 0 values.

Would it be best to go forward with the second approach and manipulate the features, so they will never be 0 values if data is present? I am still unsure of that approach because I will have to train the model with all permutations of data being present.

Is there a better way to approach this problem?

  • $\begingroup$ What exactly is the purpose of your model? I am confused by the information you have provided. $\endgroup$ Jan 10, 2023 at 15:34
  • $\begingroup$ My model's goal is to take data from multiple sources that all relate to a common instance (ie. data source 1 gives me features x and y and data source 2 gives me features a and b about the instance), and do a classification. The problem is, sometimes I will have data from some of those sources but not others. $\endgroup$ Jan 18, 2023 at 1:39

2 Answers 2


You have three options in this situation:

  • First option is to train separate models for all the possible combinations. This is a good option in case there are a lot of entries that do not have all the features.
  • Second option is to delete all the samples that do not have all features and then train the model on the ones that have all features. This is a good choice if there are not a lot of samples with missing data.
  • Third option is to impute the missing values. It is basically similar to what you have suggested with using 0, however does not affect the model that much. General practice is to replace the missing values with the mean/median/mode of the column.
  • There are also more complicated methods to do this, including running a model on the existing features to predict the missing ones. For example, if you have 3 existing features: A, B, C, and one missing feature X, you can predict X based on the values of A, B and C. You can do this by using IterativeImputer or KNNImputer from the SciKit learn library in Python. The first one uses a regression while the second uses a KNN cluster.

You may impute the missing values, as Nemo_the_scientist has recommended, and to add information to the model, to distinguish the imputed values from the original values, you could add variables that signal that they are imputed with a boolean value.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.