1
$\begingroup$

Assuming that:

  1. the problem lies in the field of natural science, i.e. relationships between variables are physics-based and does not change depending on context
  2. its a regression based model

Would it be right to assume that kernelized approaches (e.g. SVM) would perform better for unseen combinations of predictor variables, when compared to neural networks etc?

As I understood, many ML models generally fail to provide accurate prediction when the new inputs are out of distributions they were initially trained on. E.g. tree based methods, like random forest, while providing excellent output on the trained combinations of predictors fail if new variables are out of bag. On other hand,kernels ( especially the linear) project decision boundaries beyond the space of the initial training points. So assumingly, kernel-projected thresholds/boundaries help to maintain a better accuracy for unseen combinations.

$\endgroup$

1 Answer 1

1
$\begingroup$

This is a rhetorical question. You cannot say a certain model would perform better without looking at the data and it's distribution. For example if majority of the data has a linear relationship with the label then Linear Regression or linear models "might" work better.

Usually the best strategy is to try all models on the dataset and see which performs better!

$\endgroup$
1
  • $\begingroup$ Thank you. Yes, I know that the question sounds a bit naive. Ive added a bit more details to the question, tough Im not sure if it still explains what the assumption behind $\endgroup$
    – tabumis
    Sep 30, 2021 at 18:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.