What to do about a predictor with a feature importance so high above the others that it is the only determining factor in my machine learning model?

Question

I created a logistic regression model with scikit-learn which predicts the outcome of an NFL football game. It predicts the result based on features such as the team's record, opponent's record, pass yards, rush yards, etc.

I created the model and calculated the coefficients of each predictor and found the team's record and the opponent's record have huge influences on whether the team wins or not. In fact, whoever has the better record, the model will predict them to win no matter what their other features say.

Here is a graph showing the coefficients of each of my predictors, ordered by importance:

I know that record is important in determining the result of a football game, but there are plenty of factors that also should have an influence.

The first idea I thought of was to decrease the weight of the team's records so that other predictors would come into play. However, I don't think that this is the right thing to do. Is there a better model for this problem or should I get more data?

Any ideas?

Answer 1

With the info provided, as a generic approach I would have a look at:

• data quality: is your data complete, reliable and with the info you would expect with not so many missing values? Sometimes the clue is the data quality before going on modeling
• did you have a look at the correlation of each input feature with the target of interest? It might be that, before modeling and from a model-agnostic perspective, you find out that very high correlation also with few or just one input feature; in case it is weird from your expert knowledge, I would check again for data reliability or, maybe you find out that your prior belief was wrong
• try out another type of algorithms: maybde a simple decision-tree based algorithm can give you some different feature importances (now from a model perspective)
• one more precise step could be not only feature importance, but predictions explainability on a row-base (how much each value of each feature impacts your predictions): source of info