I’m currently taking a course on ML and part of my final grade is my position on a Kaggle competition (private one) regarding a classification task. The majority of groups tend to have a similar public score but there are some that spike through the rest. I am left wondering what techniques data scientists employ to increase their scores.
I am aware that a thoughtful pre-processing of the data is of chief importance as well as techniques such as grid search which help us find the best hyperparameters. This question is more about some out-of-the-box techniques that you employ when your aim is to increase your final score, for example, in the setting of a Kaggle competition.