I am working on a toy project for insurance claim prediction. In the input data for one of the feature (numeric data type) half of the values are missing. My target variable is binary which indicates if a claim was filed or not. To begin with, I imputed missing values with Mean value of feature column and then with Median value. My classification models (Logistic Regression and SVC) gave almost the same prediction metrics (accuracy, precision, recall and F1-Score) for these two imputations strategies. The third strategy that I tried involved imputing the missing values with the Mean value of each of the two categories of the target variable.

dataframe['Feature'] = dataframe['Feature'].fillna(dataframe.groupby('Target Feature')['Feature'].transform('mean')) 

After this step, the prediction metrics of my models increased considerably (5-6%). Now I am wondering if this step is correct or if the model is overfitting the data. Below are the metrics details:

Strategy: Replace Missing Values with Mean Value

Model: Logistic Regression

Accuracy:88.47% Precision:86.87% Recall:88.47% F1 Score:87.67%

Model: Support Vector Classifier

Accuracy:90.66% Precision:91.46% Recall:88.94% F1 Score:90.18%

Strategy: Replace Missing Values with Mean Value of the target feature categories

Model: Logistic Regression

Accuracy:96.44% Precision:95.06% Recall:97.69% F1 Score:96.36%

Model: Support Vector Classifier

Accuracy:96.88% Precision:97.20% Recall:96.31% F1 Score:96.75%

  • $\begingroup$ To understand if you are overfitting or not, you need to compare error on training and validation dataset, so please include them in your question. Also, dont forget that you should impute missing values only based on training dataset to prevent data leaks. If you can, please include sample data set which was used in training and feature imputing. $\endgroup$ – Yaroslaw Homenko Jan 18 at 7:49
  • 1
    $\begingroup$ You are leaking your Target. That's why it is improving. Also, why are you insisting so much on NAN Imputation? No technique will improve the score magically. You should focus more on Features, Modelling, etc. $\endgroup$ – 10xAI Jan 18 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.