# How to classify using incomplete features

Assume we have some features

pressure, volume, temperature, intensity, mass, size, ...

The problem is that, I do not have allways a complete set of these info. I can not put zero for the unknown featurs because it has a meaning. For example if I do not know the temperature I can not set it T=0. In other word how to make a difference betweem T=0 and T=unknown

It sounds like you're Dealing with Missing Data. Missing data can be handled in two ways: (1) delete values (rows or columns) or (2) perform imputation to replace missing values. Before identifying a strategy to handle this missing data, you should identify how the data is missing.

Missing Completely at Random (MCAR) means there is no difference in the observations with and without the missing values. In other words, the probability of the data being missing is the same for all classes (in a classification problem).

Missing at Random (MAR) implies that there is a systematic difference in the observations with and without the missing values, but that the difference is related to some of the observed data. Perhaps temperature is unlikely to be measured if you have pressure and volume, for example.

Missing Not at Random (MNAR) mean the probability of value being missing varies for unknown reasons.

Once you have diagnosed the missingness, you can select an appropriate technique accordingly. This may be removing the data, imputing the data, or a combination of both (i.e. removing one column and imputing another).

Removing Data

Pairwise deletion can be applied to delete all rows with missing values. Training will only occur on rows with complete data. This will introduce bias if the data is not MCAR and will not allow you to make predictions on future data that contains missing data.

Alternatively, entire features can be dropped if they have a high percentage of missing values. The effect of this on the model should be explored.

Imputation

If you are interested in retaining all of the data, imputation can be applied to fill missing values using a strategy. Sklearn has a SimpleImputer that has 4 strategies to choose from when imputing a column: filling with the "mean", "median", or "most frequent" value in that column or applying a "constant". By using constant, for example, you could represent unknown values with the string "unknown". I noticed that you mentioned numerical values in your question. For the numerical variables, adding a new column to indicate missing data might be appropriate (depending on the pattern of missingness). This could be useful as an MNAR imputation technique.

Interpolation can be used as an imputation technique to extrapolate the missing data as a function of the other observations. You did not mention what you are trying to model, but based on the features you provided, this might be appropriate.

If you are working with time series data, you should explore imputation techniques specific to time series problems such as Last Observation Carried Forward (LOCF) which replaces the missing value with the last observed value.

Other Resources

You can also look into K-Nearest Neighbors as an imputation method. It can identify the most frequent value among K neighbors, where nearness is based on distance between observed values. Be careful using this on data that has a high number of binary features. Another topic to explore might be multiple imputation for a more advanced technique.

You can actually do 3 things:

1. Use algorithm like LGBM which can ignore missing values (if you have low percentage of missing values).
2. Impute missing values with whether mean, modus or median. Try them all and measure the performance, pick one that score the best.
3. Make the missingness of your data as feature, in this notebook, the nan value count is summed and become a new variable in the data.

Some learning algorithms can handle missing values as such, i.e. they do not treat them the same way as regular values. Practically they ignore them but they can take other regular values present in the instance take into account.

The Weka ML toolkit can deal with missing values properly.

You may want to do principal component analysis to make sure there are no significant correlations in your data (for the missing values). What I mean is that if Temperature and Pressure are correlated (one is always high when the other is low), but when temperature is missing you just put it as medium, then you may be introducing a bias for that data.