how to chose between these models?

Question

i have a regression problem so i tried some regression models in order to pick the best one (based on RMSLE) here are the results:

here are all the models = [ ('LR', LinearRegression()), ('Ridge', Ridge()), ('Lasso', Lasso()), ('ElasticNet', ElasticNet()), ('Poly', PolynomialFeatures()), ('KNN', KNeighborsRegressor()), ('DT', DecisionTreeRegressor()), ('RF', RandomForestRegressor()), ('GBM', GradientBoostingRegressor()), ('XGB', XGBRegressor()), ('LGBM', LGBMRegressor())]

The question i have is how do i chose which model i will focus on ? I am a beginner in machine learning so please be patient.

Answer 1

One thing you should watch out for when applying a whole bunch of models to the same data, is your selection bias. Basically, by training these models on all of your data and picking the best one, how do you know that you didn't just pick the best model for that particular data? This is called overfitting. You have no idea if this will be the best general model if you were to give it new data.

What you want to do is come up with a procedure that will allow you to evaluate these models on subsets of the data. This is generally done through cross-validation. You divide your data into a test set and a training set. Then, you run cross-validation on your training set. Cross-validation separates the training set into k-folds, where each k is a some percent of the data. So, for example, you might run a 5-fold cross-validation. So, you can now pick a model and for each fold train on k-1 of the folds, and evaluate each model on the held-out k-fold. This will allow you to evaluate relative performance on each model and its parameters. Then you test the best model on the test set you original held out to get an unbiased estimate.

See: https://scikit-learn.org/stable/modules/cross_validation.html