I have created two simulated random dataset of 3 classes. Only difference between the dataset is that frequency of the classes.
Dataset A: (Class 0 = 300, Class 1 =200, Class 2 = 500)
Dataset B: (Class 0 = 500, Class 1 =500, Class 2 = 500)
Both are random dataset so I should expect from Logistic regresssion model to confuse between each class with equal frequencies. That means in the normalized confusion matrix I should expect equal fraction of confusion among all three classes.
Confusion matrix of Dataset A
Confusion matrix of Dataset B
My expectation from dataset A is same as dataset B. But I am not able to achieve that. why? I am using the following command in python to run the logistic regression model.
log_reg_model = LogisticRegression(C=1,penalty='l1',multi_class='ovr',class_weight='balanced',solver='liblinear')
pipe=Pipeline([('StandardScaler',StandardScaler()), ('logistic_regression',log_reg_model)])
Edit: I am uploading the both dataset in following dropbox link. https://www.dropbox.com/sh/pkiapvqy3k3f12v/AADpeBJ0XTWA2v9MCjALBcexa?dl=0 First column is index, second column is class id, third to fifth are class features.