2
$\begingroup$

I am reading a csv file with pyspark to extract some information from it. I am running pyspark locally and I do not need distributed computation. On the contrary I need some plotting library such as matplotlib to plot data. I know there are some workarounds to do it with pyspark but I found it very complicated.

My question is: is there any alternative to pyspark, possibly python-native, that allows to query a csv file allowing to use python libraries?

$\endgroup$
1
2
$\begingroup$

pandas can do it. It can read in a csv file with a single line and put the data in a dataframe that you can manipulate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.