I was looking at keras source [here](https://github.com/keras-team/keras/blob/85fe6427a50f6def7777413a4bc8b286f7ab8f8d/keras/backend/tensorflow_backend.py#L2753) which calculates cross entropy loss using:

    output /= tf.reduce_sum(output,
                            reduction_indices=len(output.get_shape()) - 1,
                            keep_dims=True)
    # manual computation of crossentropy
    epsilon = _to_tensor(_EPSILON, output.dtype.base_dtype)
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    return - tf.reduce_sum(target * tf.log(output),
                           reduction_indices=len(output.get_shape()) - 1)

`target` is the truth data, which is 0 or 1, and output is the output of the neural net.

So it looks like the loss is of the form

$$J_{y'} (y) = - \sum_{i} y_{i}' \log (y_i)$$

where $y_i$ is the model output for class $i$, and $y_i'$ is the truth data.

Does this mean the errors for $y_i' = 0$ do not contribute to the loss?  Why isn't the formula

$$J_{y'}(y) = - \sum_{i} ({y_i' \log(y_i) + (1-y_i') \log (1-y_i)})$$

used?