0
$\begingroup$

I am writing a TensorFlow program which is trying to categorize a heavily skewed dataset between two categories. One category is represented at 30x the rate of the other.

Category 0: 800
Category 1: 30000

The label for each record is a one-hot vector of length 2, representing the two categories. Using a cross entropy cost equation doesn't work very well since a very high accuracy can be attained just by rating everything as category 1.

I then made a balanced dataset that has the following data:

Category 0: 800
Category 1: 800

This dataset has about 67% accuracy using softmax, but there is a real-world cost when falsely categorizing a category 1 item as category 0, which is blown way up when mistakenly categorizing many more items as category 0 mistakenly.

I wanted to fix this by making the cost equation penalize false categorization into category 0 more than other penalties. However, my cost equation is not producing any different results than the original cross entropy equation. I think I may have programmed it in wrong, but I am unsure where the mistake is.

In theory, the skewed equation will add 1 to the normal cost when the label for a record is [0,1] and the predicted value is [1,0], which represents a miscategorization of a category 1 item into category 0.

The skewed cost equation is in the condition where FLAGS.cost == 'skewed':

x = tf.placeholder(tf.float32, shape=[None, data_length])

W = tf.Variable(tf.zeros([data_length, 2]))
b = tf.Variable(tf.zeros([2]))

y = tf.nn.softmax(tf.matmul(x, W) + b)

y_ = tf.placeholder(tf.float32, shape=[None, 2])

if FLAGS.cost == 'cross_entropy':
    cost = -tf.reduce_sum(y_*tf.log(tf.clip_by_value(y,1e-10,1.0)))
elif FLAGS.cost == 'skewed':
    positive_cutoff = tf.constant([0.5, 0.5])
    desired_tensor = tf.constant([False, True])
    true_tensor = tf.fill(tf.pack([2, tf.shape(y)[0], 2]), True)
    cost = -tf.reduce_sum(y_*tf.log(tf.clip_by_value(y,1e-10,1.0)))+tf.to_float(
        tf.reduce_all(
            tf.logical_and(
                tf.pack([
                    tf.logical_and(tf.greater_equal(y_, positive_cutoff), desired_tensor),
                    tf.logical_and(tf.less_equal(y, positive_cutoff), desired_tensor)
                ]),#shape [2,x,2]
                true_tensor#shape [2,x,2]
            )#shape [2,x,2]
        )#shape [1]
    )

train_step = tf.train.GradientDescentOptimizer(0.01).minimize(cost)

correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_,1))

accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
$\endgroup$
  • 1
    $\begingroup$ Welcome to DataScience.SE! Here's an example of a skewed cost function for regression in tensorflow. $\endgroup$ – Emre May 23 '16 at 21:51
0
$\begingroup$

The equation I ended up using is listed in the link below, provided by Emre:

Linear regression with non-symmetric cost function?

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.