Short version: despite lots of reading, machine learning still feels like being a monkey in the dark. Any advice?
For background, I'm a researcher in computer science, in a field non-related to machine learning.
I have been trying to get more proficient in machine learning*, yet no matter how much I read and fiddle with code/toy datasets, when I try to go to a harder problem, I always feel overwhelmed by the choices I need to make:
- I have to choose the algorithm: This is the part I typically find the most straightforward;
- For said algorithm, I have to choose the objective function : usually, many are applicable, and I find it difficult to gain a good intuition of what makes an objective function adapted in some cases rather than others, apart from the very classical ones for linear or logistic regression
- And then, I should devise the features: this still feels completely arcane to me, apart from using content-based features readily available in the data.
I am under the impression that I have to "create" the tailored algorithm and the data.
Concerning the algorithm, I have spent some time into studying gradient boosting and the math behind it, to the point that I have a reasonably solid comprehension of how it works, and an intuition of parameter tuning for simple datasets. However, that knowledge does not generalize.
How are these issues typically approached? Are there any resources that can help?
* By taking the Machine Learning Coursera course and its more in-depth version, reading more XgBoost-specific material (on its internals and parameter tuning and intuition), as well as playing with the Titanic dataset, and a housing market dataset.