I have 2 classes of data A and C. I want to create a NN with 3 output nodes, classifying data as either A, B, or C. Classifying an A or C correctly should have zero cost (very good). Classifying an A as a C or vice versa should have high cost (very bad). However, for samples that are "blurry", I'd like to classify them as B, which would have relatively low cost (better than a misclassification, but still not ideal). I am new to tensorflow, and am not comfortable with the different ways to define cost functions. Can anyone point me in the right direction on this one? Thanks in advance
$\begingroup$
$\endgroup$
$\begingroup$
$\endgroup$
You are talking about having different missclassification costs.
Classification consists of outputing a continuous score which is then thresholded to decide the discrete classification. The threshold (ex. probability>0.5 leads to C) is chosen to satisfy your apetite on sensitivty/specificity.
The training algorithm only cares about your continuous score. This means that you can make it easy for yourself by doing simple binary classification (typically with softmax_cross_entropy_with_logits-loss) and create your vague class B after training is done by thresholding say [0,0.4) is A, [0.4,0.6] is B and (0.6,1] is C.
To achieve exactly what you're saying, penalizing certain errors more, then you want to use the weighted_cross_entropy_with_logits. This throws out the interpretation of your outputted score as a probability, and you will need to work more to get a reasonable threshold afterwards so this is not recommended as a first step as it will probably add more complexity than results.
I also recommend reading the wikipedia article on roc analysis
