I know similar questions have been asked and I already looked through a lot of them to figure out what the issue might be, but to no avail.

I am a College student currently working on a Big Data Project. I have to setup 5 Computers (1 Master + 4 slaves) to run Apache Spark on YARN. I had an initial Hadoop setup that I setup Spark on but I messed on some steps as the guides were from different sources.

So, I decided to start fresh with a guide for both Hadoop and Spark from the same source. I found and followed this Hadoop Setup Guide.

Note: Since the PCs have a good amount of Memory, I did not Configure Memory Allocation and left it as is.

After setting Hadoop up, I tried to start the daemons with start-dfs.sh and start-yarn.sh.

On the master Node, jps showed this:

21922 Jps
21603 NameNode
21655 DataNode
21787 SecondaryNameNode

but, on all the slave nodes jps came up with nothing:

19819 Jps

Is there something that I could have messed up here or is there an error in the Guide?

I had setup Hadoop previously and use this guide.

I tried some solutions that I found online but was unable to fix the issue. For reference, the Spark Installation Guide though I haven't started with this yet.

I would really appreciate any and all help. Thanks!



Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.