Im new to Big Data, so please just dont blast me.
Can you explain me why HDFS works best when the file blocks are evenly spread across the cluster ?
Data Science Stack Exchange is a question and answer site for Data science professionals, Machine Learning specialists, and those interested in learning more about the field. It only takes a minute to sign up.Sign up to join this community
If the data is not evenly distributed, there are several data blocks that take substantially longer to perform certain queries which undermines the performance of the whole Hadoop cluster and can even lead to errors.
Since HDFS is built to guarantee high fault tolerance, it needs to ensure balanced file blocks to minimize the risk of failure.