Filling na values with condition from other column

Question

I am working on famous titanic dataset and I want to replace na values in Age column but on such a condition that these people whose Pclass=3 receive 25, Pclass=2 29 and Pclass=1 38. I was trying to do it with df.loc and df.fillna without luck. Can anyone help?

my most hopeful piece of code was it

for i in range(0,len(train_data_clean)):
if (df.loc[i][2] == 3) & (math.isnan(df.loc[i][4])):
    df.loc[i][4] = 25
if (df.loc[i][2] == 2) & (math.isnan(df.loc[i][4])):
    df.loc[i][4] = 29
if (df.loc[i][2] == 1) & (math.isnan(df.loc[i][4])):
    df.loc[i][4] = 38

but this also did not work

Answer 1

You can use df.apply to make a series containing the substitution values, then use df.fillna with the new series. Here's an example:

# map from Pclass value to replacement value
swaps = {
    3: 25,
    2: 29,
    1: 38,
}

# create series of substitution values
substitutions = df.apply(
    lambda row: swaps[row["Pclass"]],
    axis=1,  # apply to each row
)

# fill NA in Age column with substitutions
df["Age"] = df["Age"].fillna(substitutions)

Keep in mind that df.apply is very slow compared to most operations in pandas because it is not vectorized. But for a dataset of only ~900 rows, that doesn't really matter.