# How to justify a predictor in influencing the outcome?

I am working on a prediction (binary classification) problem

Currently I get an AUC score of 85-86 and F1-score of 81

Questions

1) The above performance is based on 6 well-known features

2) Let's say I add another feature and I see my AUC and F1 score improve by 1 or 2 points? So is this the only way to know that it really helps/value addition to the model?

3) How do I justify that these features really help in predicting the output?

4) Is there anyway to prove or validate that adding this feature really helps my model and improves the outcome? Is it only using changes in AUC score?

5) For instance, I can add multiple features and my AUC increases by few decimal points, so can I say that they are important or useful or drive predictions? Yes ofcourse, but adding 10 features (where 6 features really impact the outcome and rest 4 increase the auc only by few decimal points). doesn't really overfit. Am I right? Because my prediction score doesn't go beyond 86

• Hi, thanks upvoted. Yes, I understand. what I am trying to know is how do I say that addition of feature is useful? Is it only through improvement in my metric like F1 score or AUC score or is there any other approach to validate the usefulness of a feature?. If my metric goes up, I don't have to worry about anything and can claim that this new feature helps the model better. Am I right to understand this? – The Great Dec 27 '19 at 9:46
• Or is it by what you suggested yesterday through causal approaches like CFE and PFI? – The Great Dec 27 '19 at 9:50
• Now I am not sure which additional features to retain and drop. I mean adding few features increases the F1-score by decimal points or sometimes even by 1 whole point. So whether it is useful to add these features for this slight improvement. As my model AUC is only at 86, adding features (2-3 FEATURES) to take it to 87 doesn't really make the model complex. Am I right? – The Great Dec 27 '19 at 10:09