Comparison of machine learning approaches for a topic in a scientific paper

Question

As part of my master's thesis, I have made a prediction of data with approaches of machine learning in a topic where are no papers yet. The topic is a regression problem for which several machine learning approaches have been tested: SVR, RandomForestRegressor, GradientBoosting, DecisionTrees, Artificial Neural Networks, LSTMs, K-NearestNeighbors..

It has been shown that useful results can be achieved with all approaches. Which of the methods is the best depends heavily on the parameter configuration.

It would be too much work to test any parameter configuration for any approach. Besides, I think it would be uninteresting to show dozens of plots about the parameter tuning tests, because they also depend heavily on the respective data sets.

So how do you typically compare different types of machine learning approaches in scientific papers that are heavily dependent on the parameters? Just play a bit with grid search and randomized search until you are satisfied with the parameter configuration?

Answer 1

If I understand, you could evaluate your approach based on your efforts in using the right preprocessing, picking the right features, building the appropriate architecture (choosing the model that best suits the problem).

Each step will make the loss decrease until you reach a point, you will have to search over the hyperparameters to squeeze every percent of accuracy, which is heavily a computational problem.

Answer 2

Hello I don't know about scientific paper.

But I know that to compare quality of ML algorithms one against another you need to establish a metric : Make it a square mean error or anything else, a final number that can tell you if your algorithm is working as expected.

On a side note : Every algorithm, if used in their correct purpose, is highly parameters dependent. So I'm not sure what you're referring to with "machine learning approaches that are heavily dependent on the parameters"