I am trying to figure out which pyspark library to use with Word2Vec and I'm presented with two options according to the pyspark documentation.
mllib seems to be for using RDD's. And
ml seems to be using "DataFrames".
What is the difference? Shouldn't they both be using RDDs if this is spark under the hood?
What is a "DataFrame" here? As the documentation doesn't explain it.