I have been searching for methods to plot in PySpark.
I couldn't find any resource on plotting data residing in DataFrame in PySpark. The only methods which are listed are:
- through method
collect()
which brings data into 'local' Python session and plot - through method
toPandas()
which converts data to 'local' Pandas Dataframe.
The problem is that these both are very time-consuming functions. Is there any method by which we can plot data residing in Spark session directly (not importing it into the local session)?