Increasing/Decreasing importance of feature/thing in ML/DL

Question

I have 3 cases:

1. I have a classification model that will be used to classify cats and dogs. On my train data dog pictures has a watermark on them, but cat pictures don't. The problem is: Whenever I have a watermark on a cat picture, the model will predict the cat picture as a dog picture
2. I have another classification model that classifies questions and normal sentences. But whenever I have the "how" word in my normal sentence, the model will classify it as "question"
3. I have a prediction model. I have 5 columns but column number 3 is very important. I mean the importance of that column is very high. But my model cannot understand it.

All of those problems have 1 common problem. The importance of "something" or "feature" is being misunderstood by models. How these kinds of problems can be solved?

Answer 1

I would not say these models "misunderstand" anything. They simply learn from the data provided based on their inductive biases. I hypothesize that all three cases might be caused by the chosen (train) datasets:

• In case 1 the train data is not representative of the test or deployment data since only the train data has watermarks on an image if and only if it shows a dog. If that is not the case for your test or deployment data then you need to adjust your train data accordingly to remove this artifact.
• In case 2 I suggest to check the distribution of the word "how" in questions and non-questions in your train data. It might be that "how" almost exclusively occurs in questions which would, again, be a problem with the dataset not stemming from the same distribution as the data you run inference on. If that is not the case I'd check if your model cannot differentiate between the word "how" appearing in in a question vs. a non-question. If that is the case, a different model type might be more suitable.
• In case 3 it is unclear to me how you derive the importance of "column 3". It might, again, be a problem with the train dataset which, in this case, might not present that feature as very important. Alternatively, it could be that the chosen model is not able to learn the association between that feature and the target (simple example: there is a non-linear association but the model is linear).

In summary, case 1 can be handled by feeding a different train dataset. Case 2 and 3 might require different train datasets but might alternatively require different models if the problem is not with the datasets.