2
$\begingroup$

I have been looking for a comprehensive alternative to Apache Spark for Big Data analytics/machine learning and couldn't find one. The ones which I have come across are:

Can any of these can be considered as a complete alternative/better than Spark? Or Spark is the only comprehensive Big Data analytics/machine learning engine there is these days?

$\endgroup$
4
  • $\begingroup$ Can you tell us why you need an alternative for Apache Spark? $\endgroup$
    – aleck
    Dec 23 '18 at 8:56
  • $\begingroup$ The ray project ray.readthedocs.io/en/latest github.com/ray-project/ray $\endgroup$ May 1 '19 at 15:32
  • $\begingroup$ I look for alternative, because Spark is not superb performance-wise. I mean, when you created a "data preparation pipeline" - it may be OK for offline training, but you hardly can use the same pipeline for scoring. For the model speedup there is a separate product called "Mleap" - able to convert the model in something fast. But for the rest of the pipeline I see not performance hope.Strange...Also strange that best ML libs written in slower python, not in scala/java. $\endgroup$ Jul 3 '19 at 14:28
  • $\begingroup$ Have you looked into Google BigQuery? $\endgroup$
    – ttfreeman
    Aug 31 '19 at 4:15