4
$\begingroup$

Short version: despite lots of reading, machine learning still feels like being a monkey in the dark. Any advice?

For background, I'm a researcher in computer science, in a field non-related to machine learning.

I have been trying to get more proficient in machine learning*, yet no matter how much I read and fiddle with code/toy datasets, when I try to go to a harder problem, I always feel overwhelmed by the choices I need to make:

  1. I have to choose the algorithm: This is the part I typically find the most straightforward;
  2. For said algorithm, I have to choose the objective function : usually, many are applicable, and I find it difficult to gain a good intuition of what makes an objective function adapted in some cases rather than others, apart from the very classical ones for linear or logistic regression
  3. And then, I should devise the features: this still feels completely arcane to me, apart from using content-based features readily available in the data.

I am under the impression that I have to "create" the tailored algorithm and the data.

Concerning the algorithm, I have spent some time into studying gradient boosting and the math behind it, to the point that I have a reasonably solid comprehension of how it works, and an intuition of parameter tuning for simple datasets. However, that knowledge does not generalize.

How are these issues typically approached? Are there any resources that can help?

* By taking the Machine Learning Coursera course and its more in-depth version https://www.youtube.com/playlist?list=PLA89DCFA6ADACE599, reading more XgBoost-specific material (on its internals and parameter tuning and intuition), as well as playing with the Titanic dataset, and a housing market dataset.

|improve this question|||||
$\endgroup$
  • $\begingroup$ Sky is the limit... $\endgroup$ – Aditya Mar 9 '18 at 5:05
  • $\begingroup$ Like any other thing: practice. Just go and create ML systems that solve actual problems and learn what you need to solve those problems. In the process, you will gain knowledge, experience and intuition. $\endgroup$ – ncasas Mar 9 '18 at 11:00
2
$\begingroup$

We have to climb up a steep learning curve when we learn about machine learning. Your question is quite general: One of the tactics I use when learning is divide and conquer. Get some coarse overview about the whole area, then pick some particular area and dig deeper only there.
Perhaps the question is too general, the best tactic may vary and depend on the area you address. But I am not sure if learning the math is always helpful (although it may always be interesting for those who care).

The algorithms can often be applied in a black box approach, and it may be sometimes not necessary to understand an algorithm in math terms (white box), but sufficient to know it's function, strengths and weaknesses (black box).

You may be the first one that tests that algorithm for the domain, so pure experimentation is useful in the end.

|improve this answer|||||
$\endgroup$
  • $\begingroup$ I agree. You might want to focus on neural networks first. Since you are a computer scientist, the implementations should be quite accessible for you; and I always had the impression that data preparation is more standardized for neural networks. $\endgroup$ – Elias Strehle Mar 9 '18 at 22:28
  • $\begingroup$ Thank you for your replies; I find it hard to use the algorithms as black boxes (probably because of my training and research area), I will try to get better at it. :) Thank you Elias for the neural networks advice, it is unexpected. I have mostly avoided neural networks for now, under the impression they were less accessible than statistical methods. $\endgroup$ – Tiphaine Mar 14 '18 at 10:58
0
$\begingroup$

The road to Machine Learning enlightenment is highly non-linear.

You already took a great step, which is coming to Data Science exchange to ask questions. :) This is a great place, full of resources and good pointers.

As mentioned before, you need to slowly pave your way through different topics, from practical decisions about datasets, computer, training time, to deeper decisions about the model and overall approach.

My recommendation as someone trying to go through the same path is to look for great leaders in the area and go through their courses thoroughly. The first one I can recommend is David MacKay, with his free book (http://www.inference.org.uk/itila/book.html and lectures (http://videolectures.net/david_mackay/). David's approach to Machine Learning is that of a "connector", trying to show connections between different fields. Take a look at his lectures and try to follow them with exercises on the book. This will give you a solid foundation on which to draw upon when you are in trouble. He saved me more than once.

Another recommendation is Andrew Ng's Coursera course on Machine Learning. Although not as deep, it is full of practical advice for different methods.

Having a good understanding will strengthen your foundations so when tides, such as Deep Learning, come towards you, you will know how to keep your stand and make the most of it. You will understand how tides come and go and allow them to transform you instead of overwhelming you.

|improve this answer|||||
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.