Can I use sklearn's KNN imputer to fit the model to my training set and impute missing values in the test set using the neighbours from training set ?
Is it allowed ? Or , Should I only fit and transform both on a single set ?
$\begingroup$
$\endgroup$
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
The short answer is that usually if you are performing data transforms, you fit on the training data and transform training and test data.
The long answer is that if you are doing purely supervised machine learning then you want to avoid "leakage".
Data leakage is when information from outside the training dataset is used to create the model.
I'd read the above link if you want more information. In your case, it just comes down to preprocessing only using information in your training data.
