Skip to main content
Data Science

Return to Question

added 6 characters in body; edited tags; edited title
Source Link
Green Falcon
Green Falcon
  • 14.2k
  • 10
  • 58
  • 98

Is there a conventional flowchart for deciding which regression/classification techniques to useany consensus on choosing an appropriate ML approach?

I am studying data science at the moment and we are taught a dizzying variety of basic regression/classification techniques (linear, logistic, trees, splines, ANN, SVM, MARS, and so on....), along with a variety of extra tools (bootstrapping, boosting, bagging, ensemble, ridge/lasso, CV, etc etc). Sometimes the techniques are given context (eg. suitable for small/large datasets, suitable for a small/large number of predictors, etc) but for the most part, it seems like for any regression or classification problem there exist a dizzying array of options to choose from.

If I started a job in data science right now, and was given a modelling problem, I don't think I could do any better than just try all the techniques I know with basic configurations, evaluate them using cross validation-validation and pick the best. But there must be more to it than this.

I imagine an experienced data scientist knows the catalogue of techniques well and follows some mental flowchart to decide which techniques to try, instead of mindlessly trying them all. I imagine this flowchart is a function of a) number of predictors; b) variable types; c) domain knowledge about possible relationships (linear/non-linear); d) size of the dataset; e) constraints around computation time and so on.

Is there any such agreed on, conventional flowchart to follow, to choose the techniques? Or does it really boil down to "try lots of things and see what works best on the desired measure eg. cross validation"-validation"?

Is there a conventional flowchart for deciding which regression/classification techniques to use?

I am studying data science at the moment and we are taught a dizzying variety of basic regression/classification techniques (linear, logistic, trees, splines, ANN, SVM, MARS, and so on....), along with a variety of extra tools (bootstrapping, boosting, bagging, ensemble, ridge/lasso, CV, etc etc). Sometimes the techniques are given context (eg. suitable for small/large datasets, suitable for small/large number of predictors, etc) but for the most part it seems like for any regression or classification problem there exist a dizzying array of options to choose from.

If I started a job in data science right now, and was given a modelling problem, I don't think I could do any better than just try all the techniques I know with basic configurations, evaluate them using cross validation and pick the best. But there must be more to it than this.

I imagine an experienced data scientist knows the catalogue of techniques well and follows some mental flowchart to decide which techniques to try, instead of mindlessly trying them all. I imagine this flowchart is a function of a) number of predictors; b) variable types; c) domain knowledge about possible relationships (linear/non-linear); d) size of dataset; e) constraints around computation time and so on.

Is there any such agreed on, conventional flowchart to follow, to choose the techniques? Or does it really boil down to "try lots of things and see what works best on the desired measure eg. cross validation"?

Is there any consensus on choosing an appropriate ML approach?

I am studying data science at the moment and we are taught a dizzying variety of basic regression/classification techniques (linear, logistic, trees, splines, ANN, SVM, MARS, and so on....), along with a variety of extra tools (bootstrapping, boosting, bagging, ensemble, ridge/lasso, CV, etc etc). Sometimes the techniques are given context (eg. suitable for small/large datasets, suitable for a small/large number of predictors, etc) but for the most part, it seems like for any regression or classification problem there exist a dizzying array of options to choose from.

If I started a job in data science right now and was given a modelling problem, I don't think I could do any better than just try all the techniques I know with basic configurations, evaluate them using cross-validation and pick the best. But there must be more to it than this.

I imagine an experienced data scientist knows the catalogue of techniques well and follows some mental flowchart to decide which techniques to try, instead of mindlessly trying them all. I imagine this flowchart is a function of a) number of predictors; b) variable types; c) domain knowledge about possible relationships (linear/non-linear); d) size of the dataset; e) constraints around computation time and so on.

Is there any such agreed on, conventional flowchart to follow, to choose the techniques? Or does it really boil down to "try lots of things and see what works best on the desired measure eg. cross-validation"?

Source Link
Brendan Hill
Brendan Hill
  • 155
  • 8

Is there a conventional flowchart for deciding which regression/classification techniques to use?

I am studying data science at the moment and we are taught a dizzying variety of basic regression/classification techniques (linear, logistic, trees, splines, ANN, SVM, MARS, and so on....), along with a variety of extra tools (bootstrapping, boosting, bagging, ensemble, ridge/lasso, CV, etc etc). Sometimes the techniques are given context (eg. suitable for small/large datasets, suitable for small/large number of predictors, etc) but for the most part it seems like for any regression or classification problem there exist a dizzying array of options to choose from.

If I started a job in data science right now, and was given a modelling problem, I don't think I could do any better than just try all the techniques I know with basic configurations, evaluate them using cross validation and pick the best. But there must be more to it than this.

I imagine an experienced data scientist knows the catalogue of techniques well and follows some mental flowchart to decide which techniques to try, instead of mindlessly trying them all. I imagine this flowchart is a function of a) number of predictors; b) variable types; c) domain knowledge about possible relationships (linear/non-linear); d) size of dataset; e) constraints around computation time and so on.

Is there any such agreed on, conventional flowchart to follow, to choose the techniques? Or does it really boil down to "try lots of things and see what works best on the desired measure eg. cross validation"?