Benchmarking mlr (default) learners on OpenML
The entire openml database of ML results.
Test from RStudio suggests SVM.
Mlmastery suggests LDA and Trial and Error.
Do we Need Hundreds of Classifiers to Solve Real World Classification Problems? by Fern ́andez-Delgado et al.
Paper concludes parallel random forest (parRF_t) is best followed by random forest, LibSVM with Gaussian kernel (svm), extreme learning machine with Gaussian kernel, C5.0 decision tree and multi-layer perceptron (avNNet).
The best boosting and bagging ensembles use LibSVM as base classifiers
(in Weka), being slightly better than the single LibSVM classifier,
and adaboost R (ensemble of decision trees trained using Adaboost.M1).
The probabilistic neural network in Matlab, tuning the Gaussian kernel
spread (pnn m), and the direct kernel perceptron in C (dkp C), a very
simple and fast neural network proposed by us (Fern ́andez-Delgado et
al.,2014), are also very near to the top-20.
Wainer, Jacques (2016) Comparison of 14 different families of classification algorithms on 115 binary datasets Based on Fernandez-Delgado et al. (2014). "We have shown that random forests, RBF SVM, and gradient boosting machines are classification algorithm that most likely will result in the highest accuracy"
Rich Caruana & Alexandru Niculescu-Mizil () An Empirical Comparison of Supervised Learning Algorithms (classification) concludes with Platt-Calibrated Boosted Trees as best followed RF BagT Cal.SVM NN.
Many other studies include comparisons of models used. Some papers prefer SVM others SVM with radial-basis or polynomial kernel for classification. (maybe same thing)
From my own regressions on generated data I recommend earth(MARS) Cubist SVMlinear.
Manisha Thesis first runs tests on UCI Machine Learning Repository then soil fertility which is the focus of the thesis. Best models on UCI were :"elm-kernel is the ELM neural network but with Gaussian kernel", "svr is the support vector machine for regression, with Gaussian kernel using the Lib-SVM library with the C++ interface", extraTrees and cubist. The thesis includes great descriptions of each model and links to more papers."extraTrees achieved
the best RMSE for 7 of 10 soil problems". Paper is definitely worth a read.