What happens when we do repartition on a PySpark dataframe based on the column. For example
dataframe.repartition('id')
Does this moves the data with the similar 'id'
to the same partition? How does the spark.sql.shuffle.partitions
value affect the repartition?