Why is it necessary to clean/preprocess data if the unseen real data is bound to consist missing values, outliers etc.?

Question

Can it really be called Generalization if we remove/modify data points to suit our model?

Answer 1

It depends. Actually there are research papers finding that neural network can sometimes cope very well with sparse or noisy labels. But basically when you apply any machine learning approach, you want your model to pick up patterns in your data. In order to be a pattern something needs to be repetitive and predictable. Outliers and errors in your data might not adhere to that. They rather break the pattern. It can still work but it is harder to recognize it.

Making a model "learn" outliers/missing values etc. will only work, if these incidences also appear in a pattern which makes them unlikely to be outliers. As a rule of thumb it is therefore a good idea to remove this noise.

Finding patterns in exceptions & errors

As I mentioned some researches try to address the noise issue for example by classifying if there is random ("white") noise or structured noise. They not only predict the target value but also the type of noise and then try to make another prediction based on both pieces of information. So there are some active approaches to address noise but usually this is due to the fact that it would be to cumbersome or even impossible to remove the noise.

Answer 2

IMO, the goal of cleaning/preprocessing is to remove the "randomness" of the impact of missing/dirty values.

Machine learning is just mathematics. And this specific set of mathematics require numbers to work with. An empty value doesn't represent a number.

For example if I take a simple linear regression example where :

${y=a*x+b}$ where ${x=\emptyset}$, what is the algorithm supposed to output?

By cleaning the data, eg. adding the mode value, you minimize the impact of the missing data on your algorithm.

Also, don't forget that missing data is often just that, missing.

If the variable height of ${x_1}$ in my... tree classifier (example) is null, it's not because the tree doesn't actually have an height, it's just because it is absent in the dataset.

So, to answer your question, we need to generalize to deal with real world problems (missing data) while having the smallest impact possible on our prediction.