Treat similar observations in a classification problem

Question

I have a dataset of about 200k rows and I'm working on a classification problem.

Grouping the dataset by a key variable, I noticed that some rows, with the same key value, have similar values in other columns (not equal). It may be useful to calculate the distance between observations and then deleting the closest ones prior to modelling?

I tried to split the dataset by key and then I calculated the euclidean distance between observations, but now I have a list of distance matrices and I don't know what to do next.

What do you think? I'm on the right way? Do you know some literature about it?

Answer 1

Wouldn't remove similar looking observations unless you have a strong reason to do so. By deleting similar looking observations you may be adding bias into the underlying distribution responsible for generating the data.

Your model may be misled into learning a biased distribution and that may affect final performance.

To start with use the entire dataset as is. If the size is a concern, then sample a random subset of the data to build your model.