I am training a model in Spark using Dl4J library in Yarn-Cluster mode. When I train the model on 2 lakh data (approx 200MB) then the job succeeds but when I go to train the model with 3 lakh data (approx 300MB) then the job finishes with a fail status and the model doesn't get saved in HDFS. So somehow I am reaching to a bottleneck over there.

I am using 9 containers with 1 core and 1.5GB memory to run the training job.

There is no error in yarn or spark logs.

How do I identify where the problem has occurred? How do I identify if it is the insufficient resources that is causing the problem or is the code that is the problem?

  • $\begingroup$ Contact Adam Gibson. $\endgroup$ – Emre Jan 10 '17 at 18:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.