1
$\begingroup$

I want to know why leaky features are problematic in machine learning/data science. I'm reading a book that uses the Titanic dataset for illustration. It says that the column body (Body Identification Number) leaks data since if we are creating a model to predict if a passenger would die, knowing that they had a body identification number a priori would let us know they were already dead.

Logical-wise, this makes sense. But assuming I don't have any knowledge about this dataset, I will just keep the body feature and build a model, say RandomForestClassifier. Even though later on I discover it leaks data, so what? As long as my test set has this column, the model still runs and still give me a prediction (a very good prediction, indeed).

UPDATE I followed a linked thread here, Why does my model produce too good to be true output? and got some more thoughts. Let's assume a very hypothetical situation: someone wants to mess with my predictions by purposely attacking my data source and alter the data in the body column. In this case, I can see that dropping it before model training makes sense since this completely fools the models. This got me into thinking that data governance is equally important, perhaps even more important, than building a good ML model.

But I rarely see this practice in real-world projects. Usually, the expectation is: given the data, train the best model. These datasets typically have thousands of feature and are already prepared by data engineers. It's very likely that they accidentally include features that make validation scores unreliably high, even if one applies tons of advanced validation techniques. This seems to make it hard for data scientists to properly train and validate models because the only way then is to comb through the data generation process, which might be perceived as unproductive.

$\endgroup$
3
  • $\begingroup$ In your example, an error would happen if a person who died was discovered later and doesn't have a body id. Also it raises the question of why doing a ML model at all: if you're going to rely on the body id as a feature to determine whether the person died, then why not just write a simple rule-based system: if has body-id then predict true else predict false. It's a matter of designing the task properly, see a similar example here. $\endgroup$
    – Erwan
    Mar 27, 2022 at 14:27
  • $\begingroup$ @Erwan Thanks for link. I updated my post. Wonder if you could share some thoughts. $\endgroup$
    – Huy Truong
    Mar 27, 2022 at 17:25
  • $\begingroup$ I agree, quite often the broad context issues like data governance are neglected. I think it's simply because the purely technical question of training/testing a model on some data is very well defined and easier to answer than other less clearly defined questions. Also it's easier to brag about performance improvement than about having thoroughly checked the quality of the data ;) $\endgroup$
    – Erwan
    Mar 28, 2022 at 15:01

1 Answer 1

2
$\begingroup$

Yes, the argument you are giving is perfectly valid. But Let look at two different scenarios and see how it does not benefit in real world:

  1. If you already have a variables which perfectly predicts the target variable why would you spend your resources on creating a ML model. Remember the objective of ML model should not be to good on Testing Data but when its deployed in production it should perform well.

  2. In real world, this leaky variable leads to bad model prediction. In some cases this variable may not be present hence your model pipeline and model performace will drastically reduce.

$\endgroup$
1
  • $\begingroup$ Thanks for the answer. I also updated my post. Wonder if you could share some thoughts. $\endgroup$
    – Huy Truong
    Mar 27, 2022 at 17:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.