My company processes Data (I am only an intern). We primarily use Hadoop. We're starting to deploy spark in production. Currently we have have two jobs, we will choose just one to begin with spark. The tasks are:
- The first job does analysis of a large quantity of text to look for ERROR messages (grep).
- The second job does machine learning & calculate models prediction on some data with an iterative way.
My question is: Which one of the two jobs will benefit from SPARK the most?
SPARK relies on memory so I think that it is more suited to machine learning. The quantity of DATA isn't that large compared to the logs JOB. But I'm not sure. Can someone here help me if I neglected some piece of information?