1
$\begingroup$

I have a model where I predict classes to define instructions for a trader robot. The classes are -2, -1, 1 and 2 (strong sell, light sell, light buy and strong buy) and I'm using a simple confusion matrix to asses the performance of the model, however I would like to know if there are other ways that takes into account the "distance" between the classes.

I've found some content about ordinal regression, but I already made my classification in other way (not sure if this content applies to me though).

Any suggestions?

Improve this question
$\endgroup$
6
  • $\begingroup$ BTW why you say „almost (?) ordinal“? For me it is just ordinal. Do I miss something here? $\endgroup$
    – aivanov
    Jan 22, 2021 at 0:03
  • $\begingroup$ Because doesn't look ordinal since there is no direction, but I'm not sure if this is a requirement for ordinal classes. $\endgroup$
    – Felipe Augusto
    Jan 22, 2021 at 1:53
  • $\begingroup$ What do you mean by direction? How to order it, i.e. using < or >? I think it doesn’t matter because it is just matter of perspective / convention. $\endgroup$
    – aivanov
    Jan 22, 2021 at 20:43
  • $\begingroup$ Yes, I see many examples like: sad, ok, happy, or 0, 1, 2, 3, 4, 5. $\endgroup$
    – Felipe Augusto
    Jan 22, 2021 at 20:52
  • $\begingroup$ I understand the instructions like strong sell = asset is significantly overpriced, sell=overpriced, buy=underpriced,etc. So basically one just encoded the difference between true value and market value, which is definitely ordered. $\endgroup$
    – aivanov
    Jan 22, 2021 at 21:01

1 Answer 1

Reset to default
1
$\begingroup$

You could just calculate the mean absolute error.

However, this quality measure (and any other measure taking into account the distance) would not be consistent with the loss function you used while training your classifier I guess. So it would not be fair to your algorithm. It is like if your professor would change the grading system after you submitted your test.

Instead of changing the quality measure you might want to reconsider your approach. The suggestion to use the ordinal regression sounds good. In case if the cost of misestimation between neighbors is similar you could even use plain regression and then round.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.