Recently I was introduced to the field of Data Science (its been 6 months approx), and Ii started the journey with Machine Learning Course by Andrew Ng and post that started working on the Data Science Specialization by JHU.

On practical application front, I have been working on building a predictive model that would predict attrition. So far I have used glm, bayesglm, rf in an effort to learn and apply these methods, but I find a lot of gap in my understanding of these algorithms.

My basic dilemma is:

Whether I should focus more on learning the intricacies of a few algorithms or should I use the approach of knowing a lot of them as and when and as much as required?

Please guide me in the right direction, maybe by suggesting books or articles or anything that you think would help.

I would be grateful if you would reply with an idea of guiding someone who has just started his career in the field of Data Science, and wants to be a person who solves practical issues for the business world.

I would read (as many as possible) resources (books,articles) suggested in this post and would provide a personal feed back on the pros and cons of the same so as to make this a helpful post for people who come across a similar question in future,and i think it would be great if people suggesting these books can do the same.


4 Answers 4


I would recommend limiting yourself to a few tried and trusted algorithms. I would not recommend Elements of statistical learning ( as a first book). It is too theoretical, aimed at graduate students, with exercises asking how to prove X or Y... I think ISL is more appropriate, with more practical advice ( in any case both books are free as pdf downloads).

Besides statistics, I would make sure you are comfortable with experimental design/AB tests, and with Business Intelligence/Visualisation.

  • $\begingroup$ Would appreciate if you could suggest some Algorithms that one should not MISS ON, or better to say are the most useful for solving practical business issues. If possible please mention the best ways to learn them (particular books,self help articles or may be trial and error) $\endgroup$
    – Vinay Tiwari
    Commented Sep 19, 2014 at 13:18
  • 2
    $\begingroup$ I would say pretty much all the algos in ISL: linear regression, logistic regression, tree based methods, SVM; Clustering and dimension reduction eg PCA. Go through the book and look at the corresponding online course ( online.stanford.edu/course/statistical-learning-winter-2014 - maybe on youtube?). $\endgroup$
    – seanv507
    Commented Sep 19, 2014 at 14:26
  • $\begingroup$ Great Resource,good to have a book and videos on the same by the Authors themselves.Thanks a lot for the link,wasn't aware of this. $\endgroup$ Commented Sep 19, 2014 at 15:03
  • $\begingroup$ I'm sorry, but this is awful advice. A data scientist should never rely on a few algorithms. You need to rely on your own skills of data and analysis and no two data problems are alike. Some will be solved with X, another with Y. It's simply not reasonable to expect the data universe to comport to your few algorithms. Be curious, be flexible, be knowledgeable and use the right tool for the job, not just the ones you happen to know. $\endgroup$ Commented Jan 9, 2019 at 19:52

Arguably someone calling themself a data scientist ought to know more about the intricacies of the algorithms they use—e.g. what affects the convergence rate of the Fisher scoring algorithm in GLM—than a common or garden statistician—who might be content just to know that the maximum-likelihood solution will be found (perhaps after they make a cup of coffee). In any case understanding the general concepts of statistics & machine learning is important in addition to familiarity with the methods you do use—the theory behind them, the assumptions they make, what diagnostic checks you should perform, how to interpret the results. Avoid being this parody.

You'd probably enjoy reading Hastie et al. (2009), The Elements of Statistical Learning.

  • $\begingroup$ Surely will read it! Really liked the last line...i think the urge and sometimes the pressure to get the results ASAP often leads to such Parodies. And its equally important to avoid the opposite of this wherein one goes so deep in the learning that it becomes useless for the real world issues. while growing/learning sometimes its more important to know what NOT to do, thanks a lot for the guidance hope to see more such insights that would enlighten me and others on a similar Journey. $\endgroup$
    – Vinay Tiwari
    Commented Sep 19, 2014 at 12:15
  • $\begingroup$ "what affects the convergence rate of the Fisher scoring algorithm in GLM"- I guess you lost 99% of the Data Scientists here. $\endgroup$
    – Momo
    Commented Sep 19, 2014 at 12:23
  • $\begingroup$ @Momo: Well, "data scientist" is one of those ill-starred terms that has barely gained currency before starting to be devalued. $\endgroup$ Commented Sep 19, 2014 at 14:40

Well, I would say knowing the intricacies of 1 or 2 algorithms in detail( like the inner workings of their parameters) is definitely better than knowing how to run a bunch of them.

I have been in the Analytics area for about 11 years and a Data Scientist for 2.5 years and I'm speaking from experience. On the other hand, you should definitely be aware of other things out there (more recent algorithms like deep learning, SVM, XGboost etc.) which might be more applicable to your problem at hand.

I think Dr. Andrew Ng's course goes into a quite a few details of some algorithms and it's a good start. As others have pointed out, http://statweb.stanford.edu/~tibs/ElemStatLearn/ is a good book and it has videos to go with it.

This is my personal opinion, the algorithms which you shouldn't miss out on are: (Know these in detail):

1) Multiple linear regression 2) Logistic regression 3) Common techniques of dimensionality reduction like PCA 4) K-means clustering 5) Non-linear regression 6) Optimization methods: gradient based search methods, linear programming and discrete optimization 7) Concepts and algorithms in feature engineering 8) Simple time-series forecasting methods

More Esoteric algorithms:

1) Random Forests 2) SVM 3) deep learning 4) Other methods of dimensionality reduction like LDA 5) Other kernel based methods 6) Genetic algorithms 7) XgBoost 8) Dynamic regression 9) GARCH/ARCH methods 10) Structural equation modeling 11) Box Jenkins methods in time-series forecasting 12) Information theory: information gain, mutual gain etc.


I had been in a similar situation. I started out with each and every algorithm here (and in great detail).

enter image description here

However, I soon found out that the academia in machine/deep learning is moving real fast, and is always coming up with faster/state-of-the-art algorithms that go a long way to outdo traditional algorithms in many real-world applications. So, it is always preferable to be updated with the latest trends. I suggest (as I normally do myself) to grab a subscription of a good newsfeed (like Medium) or an amazing, cutting-edge research journal and follow it. Many times amazing algorithms come from research papers tackling a particular problem (probably similar to yours).

The point is, to be a good data-scientist (or a ML engineer), you need a mix of both- depth and width. I personally find it useful to know a lot of algorithms on their surface (simply what they do, when are they used, pros and cons). I return to them when I feel (only feel) they might help me solve a particular problem. I read them in detail and see if they are a good fit. They might, or they might be not. But thinking about the details is essential to ensure you don't miss out on an amazing approach to your problem due to lack of insight into that approach. For instance, once I was working on something that required Object detection (very simple though). I read somewhere about R-CNN, Fast-CNN, YOLO. I immediately turned to them to see if they fit well. That day I knew them in more detail.

Whether I should focus more on learning the intricacies of a few algorithms or should I use the approach of knowing a lot of them as and when and as much as required?

Learning the intricacies is amazing. However, the world moves at a real fast pace. There might be a new algorithm that outdoes the one whom you learnt with great detail. It's time, hence, to flush out that use and see if the new one does you more good.

Learn things when needed. And when needed, learn them in detail. You should be able to apply things if you feel they can probably work. And this insight comes from knowledge.

Good luck.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.