# Loss function to maximize sum of targets

I have a dataset {$X_i$, $Y_i$}. $Y_i$ - real value targets, they can be negative. The task is to teach classifier $f: X_i \rightarrow \{0, 1\}$ to maximize the sum below $$L(f) = \sum_i f(X_i) * Y_i$$ So $f(X_i) = I[Y_i > 0]$ would be the ideal classifier. I'm looking for appropriate smooth loss function to teach NN. NN output is probability $p(X_i)$, so $f(X_i) = I[p(X_i) > 0.5]$. I have tried the next loss function: $$BadLoss(p) = \sum_i p(X_i) * Y_i$$ but because $Y_i$ can be negative it is bad-defined loss and NN converges to nothing good. The best results I got were with usual logloss: $$Loss(p) = \sum_i logloss(p(X_i), \; I[Y_i > 0])$$

Can be there better loss functions to maximize the sum $L(f)$?

• You say your targets are real, yet you want to classify them? If you have two classes use the traditional $y \in {0,1}$ assignments with the cross-entropy loss. – Emre Jan 24 '17 at 5:36
• @Emre, I don't want to classify them. Classifier for signs of $Y_i$ is just a hack that works to maximize $L(f)$. Is there a better way? I want to optimize loss that more similar to L(f). – alex_io Jan 24 '17 at 5:44

Define $Z_i = 1[Y_i>0]$, i.e., $Z_i = 1$ if $Y_i > 0$, else $Z_i = -1$. Now the data set $(X_i,Z_i)$ defines a two-class boolean classification problem. The solution to that boolean classification problem with highest accuracy solves your original problem (minimizes your loss function).
The shortcoming of this approach is that it doesn't take into account the value of the $Y_i$'s, only their sign. To improve this, use weights on the samples. We're again going to train a boolean classifier on the training set $(X_i,Z_i)$, but this time we'll weight each sample in the training set differently. Put the weight $Y_i$ on the sample $(X_i,Z_i)$ in your training set. In other words, when training the boolean classifier, the loss function will be a weighted sum of the loss for each sample:
$$\text{Loss}(\theta) = \sum_i Y_i \cdot \ell(f_\theta(X_i),Z_i)$$
where $\theta$ are the model parameters, $f_\theta(X_i)$ is the output of the classifier on input $X_i$, and $\ell(\cdot,\cdot)$ is the cross-entropy loss. Notice that errors in samples where $Y_i$ is large are penalized more, and errors in samples where $Y_i$ is small are penalized less; this is appropriate, as that's exactly what will happen when you use the classifier $f_\theta$ to compute $L(f_\theta)$. This should improve on the approach you're taking.
• Thanks for the answer. But that is exactly what I did. It worked pretty well. But maybe there is a way to take into account values of $Y_i$ not only signs. – alex_io Jan 24 '17 at 8:09