We use .cache() on RDD for persistent caching of an dataset, My concern is when this cached will be expired?.
dt = sc.parallelize([2, 3, 4, 5, 6])
dt.cache()
when does cache get expired for a RDD in pyspark?
krishna Prasad
- 1.1k
- 1
- 14
- 23