I'm looking for resources that talk about best practices or simply some examples (at specific companies or in general) on how tech companies that heavily use machine learning like Twitter, Facebook etc. set up their data science workflow to make it easy to take models from exploratory to production level.

In 2012 Twitter published a paper on what their analytics stack looks like. They heavily rely on Pig. Since this is already a few years old, my quesiton is: Is there a more recent version or something similar to that available for one of the big tech companies using data science?

A recent blog entry from Slack describes their architecture, but it says little about how it is used more specifically for machine learning purposes.


I tried to look for some resources of your interest and came up with these online courses available at eDX-

These are just the few resources, that might get your attention and may be useful for you. In addition to that, you can explore eDX - DS Dashboard for more such courses, go to Github there are lots of systematic Data Science stuff from scratch showing how things are done at professional end, setuping things up etc.

Hope it helps!

  • $\begingroup$ I've taken the last two courses and they just talk about how to use Spark. That's not what I am looking for. The first one doesn't seem to apply either. $\endgroup$ – oW_ Oct 18 '16 at 16:03
  • $\begingroup$ Then try to explore on the best Data science websites / communities like Data Science Central, Analytics Vidhya, Analytic Bridge etc. Maybe you'll get what you're looking for. Cheers! $\endgroup$ – Abhishek Jaiswal Oct 18 '16 at 16:07

Not the answer you're looking for? Browse other questions tagged or ask your own question.