0
$\begingroup$

I'm working on a project which is predicting the win rate of one team or one person. (could be any kind of sports like baseball, basketball or e-sport games) The data I have is more like a classification problem. Data looks like this, A v.s B, the winner is A. So, I trained a classification model with those data. The model should input two units(a team or a person) and output a probability from sigmoid, so I should set a threshold to determine the output is 1 or 0. But I'm curious about if the output of the sigmoid can be considered as the win rate? I think the answer might be yes? But I got next question. My goal is to predict the win rate of one unit(team or person), which means I don't care about opponents. So When using this model to predict, for example, I want to predict Laker's win rate, and I make the match-up with all the others teams(29 teams) as the opponents and make the 29 pieces data like Laker vs Hawks, Laker vs Mavericks,...and so on, then I got the 29 output from sigmoid. Here are two choices I have. 1. setting the threshold for the output of sigmoid and determine the result is 1 or 0. Then calculate the win rate. For example, 3 output, 1 0 0 ---> win rate=0.33 2. just get the mean of all the 29 games' output from sigmoid(no threshold setting). For example, 3 output 0.6, 0.3, 0.4 ---> win rate = 0.433

I think the first one is more reasonable, but the second one is outperforming the first one in my experiment. So, does the second one make any sense? For the second one, is it a regression problem? or still a classification problem? I I'm not fluent in English. Please bear with me... I would be grateful for your response.

$\endgroup$
  • $\begingroup$ The reason for the second one to outperform is very probably due to overfitting. Do you use different data sets as train/test data? $\endgroup$ – Eugen Mar 5 at 1:10
  • $\begingroup$ Yes, I do the cross-validation and get a similar result. Although the scores might be a little bit different between each fold. But the result still shows that the second one is outperforming. I use RMSE to evaluate by the way. The model is the same one. The only difference is how I use the model when predicting stage. @Eugen $\endgroup$ – user91218 Mar 5 at 1:52
  • $\begingroup$ I'd assume the correct categorization of your problem is: first one is a regression problem of classification sub-problems while the second one is a regression problem of regression sub-problems. $\endgroup$ – Eugen Mar 5 at 7:31
  • $\begingroup$ If you already used cross-validation you could play around with Bagging and Boosting data sampling-methods and compare your two models on your test data, if your second one keeps performing better then it might be the better model for your data. However, there is no model, which performs best on all possible data sets, your first model for example might perform better if you have very large data available for training or if you have noise added to the data or stuff like that. $\endgroup$ – Eugen Mar 5 at 7:31
  • $\begingroup$ If you want to improve your models accuracy, it is also important to analyze on which test cases it has the highest error results, learning to understand your data so that you can apply methods to reduce the average error by reducing errors on outliers by applying some preprocessing methods on your data before training the model. $\endgroup$ – Eugen Mar 5 at 7:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.