5
$\begingroup$

I am trying to do binary classification of News Articles (Sports/Non-Sports) using recurrent neural net in tensorflow. The training data is highly skewed [Sports:Non-Sports::1:9].

I am using cross-entropy as my cost function, which treats both classes equally.

What are the ways by which user can penalise one class? Or is there any other cost function suitable for this purpose?

$\endgroup$
4
$\begingroup$

This has already been answered both in stackoverflow and crossvalidated.

The suggestion in both cases was to add class weights to the loss function, by multiplying logits:

loss(x, class) = weights[class] * (-x[class] + log(\sum_j exp(x[j])))

For example, in tensorflow you could do:

ratio = 31.0 / (500.0 + 31.0)
class_weight = tf.constant([ratio, 1.0 - ratio])
logits = ... # shape [batch_size, 2]
weighted_logits = tf.mul(logits, class_weight) # shape [batch_size, 2]
xent = tf.nn.softmax_cross_entropy_with_logits(
  weighted_logits, labels, name="xent_raw")
$\endgroup$
1
  • 6
    $\begingroup$ I don't think you really want to multiply logits by weights but cross entropy by weights. See this answer for a correct implementation: stackoverflow.com/questions/35155655/… $\endgroup$
    – jaranda
    Mar 7 '17 at 8:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.