I estimate a multi-class origin-destination model with 45 classes. In particular the classes are geographical regions between which people can move. Currently I summarize the overall performance using Cohen's kappa to take the unequal class sizes into account. But I would love to provide deeper insights into the performance. Do you have any recommendations of how to proceed with so many classes?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
I would do a big confusion matrix and, by inspecting it, decide what classes to add to a smaller confusion matrix in order to clearly illustrate the main sources of confusion for the model.
answered Apr 12, 2020 at 20:31
-
$\begingroup$ Thank you very much for your idea! To make sure that I understand you correctly, you wouldn't change the number of classes in the classification problem but rather drop the "insignificant" entries of the confusion matrix to get a better overview? $\endgroup$ Apr 13, 2020 at 10:02
-
$\begingroup$ Yes, that’s it. You could even create a series of smaller confusion matrices, one for each group of interesting confused classes. $\endgroup$ Apr 13, 2020 at 10:18
$\begingroup$
$\endgroup$
1
you may read this journal
Abstract: Reliable prediction of short-term passenger flow could greatly support metro authorities’ decision processes, help passengers to adjust their travel schedule, or, in extreme cases, assist emergency management. The inflow and outflow of the metro station are strongly associated with the travel demand within metro networks.
My Suggestion: you might want to make an adjacency matrix and instead of 0 and 1 keep adding 1 for every match of origin-destination, and you will get a heatmap. a heat map is basically what is also known as hot spot. most traveled geographical region. now you have this heatmap 2-D array, you can plot this on a scatter plot, or on x-y axis to get more insights.
alternatively, give a shot for traditional approach. just calculate the Softmax probabilities , and you get an array for a given origin which tells the probability of next classes.
-
$\begingroup$ Thank you very much for your suggestion and the link to the journal - I will get right on it and read it :-) $\endgroup$ Apr 13, 2020 at 10:03