I am training a logistic regression model on a dataset with only numerical features. I performed the following steps:-
1.) heatmap to remove collinearity between variables
2.) scaling using StandarScaler
3.) cross validation after splitting, for my baseline model
4.) fitting and predicting
Below is my code:-
# SPLITTING
train_x, test_x, train_y, test_y = train_test_split(data2, y, test_size = 0.2, random_state =
69)
#MODEL INSTANCE
model = LogisticRegression(random_state = 69)
# SCALING
train_x2 = train_x.copy(deep = True)
test_x2 = test_x.copy(deep = True)
s_scaler = StandardScaler()
s_scaler.fit(train_x2)
s_scaled_train = s_scaler.transform(train_x2)
s_scaled_test = s_scaler.transform(test_x2)
# BASELINE MODEL
cross_val_model2 = -1 * cross_val_score(model, s_scaled_train, train_y, cv = 5,
n_jobs = -1, scoring = 'neg_mean_squared_error')
s_score = cross_val_model2.mean()
# FITTING AND PREDICTING
model.fit(s_scaled_train, train_y)
pred = model.predict(s_scaled_test)
mse = mean_squared_error(test_y, pred)
CV score is 0.06
and score after fitting and predicting is 0.23
. I find this weird as CV is a measure of how good your model performs. So I should atleast get a score equal to the CV score right?
mean_squared_error
. So 0.23 is worse than 0.06 $\endgroup$