I have implemented a prediction model. Now I am checking if I should split the model to two context-based models (men and women). I have created a new binary feature that is relevant to only part of the women's group and it can affect the prediction accuracy for this group. I don’t have a lot of data so I am not eager to split to context-based models. I know that in this cases it is best to use an interaction terms features (womenxnewfeature) but this feature is only part of the women's’ group and I have heard that adding the feature as it is it’s not good.
Any suggestion on what I can do?