I am working on a prediction (binary classification) problem
Currently I get an
AUC score of 85-86 and
1) The above performance is based on 6 well-known features
2) Let's say I add another feature and I see my
AUC and F1 score improve by 1 or 2 points? So is this the only way to know that it really helps/value addition to the model?
3) How do I justify that these features really help in predicting the output?
4) Is there anyway to prove or validate that adding this feature really helps my model and improves the outcome? Is it only using changes in
5) For instance, I can add multiple features and my AUC increases by few decimal points, so can I say that they are important or useful or drive predictions? Yes ofcourse, but adding 10 features (where 6 features really impact the outcome and rest 4 increase the auc only by few decimal points). doesn't really overfit. Am I right? Because my prediction score doesn't go beyond 86