3
$\begingroup$

Where exactly bigdata platforms fit in to a data science/ machine learning projects ?

Say I have a large dataset for a binary classification problem - cats and dogs.

Now I need to create a model for real time classification

Here is my question.

1 Since dataset is huge I can make use any distributed platform for faster computation and model creation right?

2 Once the model is ready, then there is no need of these distributed platform right ? or are they needed for feature extractions ?

$\endgroup$
3
  • $\begingroup$ I'd serve the model in something lighter, faster. Hadoop is not suited for serving models. I think you've got the idea. $\endgroup$ – Emre Mar 9 '17 at 17:38
  • $\begingroup$ so they can be used only for creating models, not for real time predictions right ? $\endgroup$ – Sreejithc321 Mar 13 '17 at 5:46
  • $\begingroup$ Spark can be near real-time, but not Hadoop. oreilly.com/ideas/… $\endgroup$ – Emre Mar 13 '17 at 15:03
2
$\begingroup$

1 Since dataset is huge I can make use any disributed platform for faster computaion and model creation right?

Yes, that is what distributed platforms are for.

2 Once the model is ready, then there is no need of these disributed platform right ? or are they needed for feature extractions ?

Yes for data extraction, feature generation for scoring and scoring your dataset (the actual reason you trained at model for!), you would still need the platform as that data is also likely to be huge.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.