3
$\begingroup$

I am working on a classification problem, using features $\mathbf{x}$ to predict a target variable $y \in \mathbb{N}_0$. By a strictly proper scoring rule I mean a loss function $\ell(y,\hat{y})$ for which given any input $\mathbf{x}$, the unique minimizer of the expected loss is $$\operatorname{arg\,min}_{\hat{y}} \mathbb{E}_{y|\mathbf{x}} \left[\ell(y,\hat{y}) \right]=\mathbb{E}_{y|\mathbf{x}}[y|\mathbf{x}]. $$ In other words, the loss incentivizes one to predict the expected value of the target variable, given the features. It is known that the squared error loss $$ \ell(y, \hat{y})=(y-\hat{y})^2$$ satisfies this property. Furthermore, if we restrict the target variable to binary classification $y \in \{0,1\}$, then the Schervish representation theorem gives all strictly proper scoring rules as integrals involving a nonnegative weight function $W$: $$ \ell(y,\hat{y}) = y \int_{\hat{y}}^1 (1-p) {W}(p) \mathrm{d}p + (1-y) \int_0^{\hat{y}} p {W}(p)\mathrm{d}p = \begin{cases} \int_{\hat{y}}^1 (1-p) {W}(p) \mathrm{d}p & y = 1 \\ \int_0^{\hat{y}} p {W}(p) \mathrm{d}p & y=0 \end{cases} . $$ For example $W(p)=2$ gives the squared error loss, while $W(p)=(p(1-p))^{-1}$ gives log loss.

I am interested in similar representations for more than 2 classes, particularly when the target variable domain is $\mathbb{N}_0=\{0,1,2,\dots\}$. Thanks.

$\endgroup$
3
  • $\begingroup$ Would you kindly add a list of explicit questions? Are you looking for a Schervish representation theorem for multiple classes? $\endgroup$
    – Ggjj11
    Aug 30, 2023 at 19:54
  • $\begingroup$ @Ggjj11 Yes, I want an analogue of the Schervish representation theorem for multiple classes. I.e. a characterization of all loss functions whose expected value is minimized at the conditional label mean. Ideally the set of labels I want is {0,1,2,...} as it contains all finite label sets as sub-cases. $\endgroup$
    – user1337
    Aug 31, 2023 at 1:26
  • $\begingroup$ As this is about mathematical statistics, the question might be a better fit on Cross Validated (Statistics) Stack Exchange, even if the topic isn’t strictly outside the purview of Data Science. $\endgroup$
    – Dave
    Oct 4, 2023 at 23:00

1 Answer 1

0
$\begingroup$

For m classes 1 vs all:

$((y(i)* \int_1^{y_i}(1-p) W(p) dp) + \sum_0 ^{ m-1}((1-y) \int_{y_i}^0 pW(p)/(m-1) dp))$

$\endgroup$
1
  • 1
    $\begingroup$ Your formulas does not make sense, please provide context and define the symbols $\endgroup$
    – Ggjj11
    Sep 1, 2023 at 21:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.