2
$\begingroup$

I'm trying to submit a python script using spark-submit on windows 7. I'm using spark 2.3.0 built for hadoop 2.7 . The python script is supposed to read data from a csv file and plot graphs. is it possible to run such a script using spark-submit? In the python script I included this block of code for spark context. 

import findspark
findspark.init()
import pyspark
from pyspark import SparkContext
APP_NAME = 'kineticPCA'
conf = SparkConf().setAppName(APP_NAME)
conf = conf.setMaster('spark://192.168.xx.xxx:7077')
sc = SparkContext(conf=conf)

This is the how submit the job on command prompt, on the spark bin directory 

spark-submit G:/guru orders/pca kinetics/files/kineticPCA.py

This is the error I get:

Error: Cannot load main class from JAR G:/guru with URI G. Please s through --class. Run with --help for usage help or --verbose for debug output

enter image description here

What could be the cause of this error?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

use this command .\bin\spark-submit \guruorders\pcakinetics\files\kineticPCA.py

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.