I have 100s of columns with binary values [0, 1] plus some extra columns without binary values. I am trying to do regression model but the model performance is very low. For non-binary features, I have used PCA to decrease the dimension of it. I don't think its appropriate to use PCA for binary values. I am guessing, its because of the large number of binary columns, the model isnt doing great. What can be done on this kind of situation ? I have tested with almost all of the regression model available in sklearn.
What approach I could take to improve the model performance ? Any suggestions.