Though tree-based ML algorithms give us 100% accuracy on train dataset many times, but why is this not happening every time. I know this results in overfitting but why not 100% accuracy every time on the dataset using which our model is trained?
- Contradictory/inconsistent data (e.g. the training data contains the same input with different outputs).
- Limits in the model's memorization capacity (e.g. limited tree depth).
If the model has infinite memorization capacity and the data is consistent, it should be possible to memorize the whole training data, as you hinted.