I have a classification problem with highly imbalanced data. I have read that over and undersampling as well as changing the cost for underrepresented categorical outputs will lead to better fitting. Before this was done tensorflow would categorize each input as the majority group (and gain over 90% accuracy, as meaningless as that is).
I have noticed that the log of the inverse percentage of each group has made the best multiplier that I have tried. Is there a more standard manipulation for the cost function? Is this implemented correctly?
from collections import Counter counts = Counter(category_train) weightsArray = for i in range(n_classes): weightsArray.append(math.log(category_train.shape/max(counts[i],1))+1) class_weight = tf.constant(weightsArray) weighted_logits = tf.mul(pred, class_weight) cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(weighted_logits, y)) optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)