I'm still a newbie in Data Science and I found the Titanic question and solution here in Kaggle. I've been trying to understand the solution yet I still can't grasp why he used those percentiles for the numerical feature distribution part.
Here's the analysis:
- Total samples are 891 or 40% of the actual number of passengers on board the Titanic (2,224).
- Survived is a categorical feature with 0 or 1 values.
- Around 38% samples survived representative of the actual survival rate at 32%.
- Most passengers (> 75%) did not travel with parents or children.
- Nearly 30% of the passengers had siblings and/or spouse aboard.
- Fares varied significantly with few passengers (<1%) paying as high as $512.
- Few elderly passengers (<1%) within age range 65-80.
Here's the code:
train_df.describe() # Review survived rate using `percentiles=[.61, .62]` knowing our problem description mentions 38% survival rate. # Review Parch distribution using `percentiles=[.75, .8]` # SibSp distribution `[.68, .69]` # Age and Fare `[.1, .2, .3, .4, .5, .6, .7, .8, .9, .99]`
These questions were asked a couple of times in the comment section of the solution yet I can't find any concrete answer. Can anyone try to explain to me in great detail the reason behind the usage of the specific percentiles?