# What is the best way to train a model?

I am trying to train my model for sports predictions.

The data frame is as a below given example:

     datetime             country    league                        home_team            away_team              home_odds    draw_odds    away_odds    home_score    away_score
---  -------------------  ---------  ----------------------------  -------------------  -------------------  -----------  -----------  -----------  ------------  ------------
0  2020-02-22 14:00:00  Albania    First Division                Dinamo Tirana        Beselidhja Lezha            4.66         3.74         1.59             2             0
1  2020-02-16 14:00:00  Albania    First Division                Beselidhja Lezha     Burreli                     1.82         3            4.42             2             1
2  2020-02-08 14:00:00  Albania    First Division                Terbuni              Koplik                      1.41         4.2          5.85             2             1
3  2020-01-26 13:00:00  Albania    First Division                Dinamo Tirana        Egnatia Rrogozhine          2.51         2.98         2.64             0             0
4  2020-01-25 13:00:00  Albania    First Division                Elbasani             Oriku                       2.36         3.2          2.66             2             0


What would be the best way to train the model for predictions?

The training data is a database of all the soccer competitions and teams.

• Should I be training the model with the competitions in the testing data (filter out all of the rest and keep the competition that the team has played before or is playing) and then predict?

or

• Keep the training data as is and predict?

Because a team has data outside the competitions as well.

Example:

Chelsea has played in the FA Cup, Champions League, Premier League and other competitions. I want to predict a Chelsea's match for the Champions League. Should I be taking training data for Chelsea for all the competitions or should I filter training data for Chelsea for just the Champions league?

What could be defined as 'noise' in such a model?

What is the most useful approach data science wise?

• U can build a model per group aka league (any tree based or NN models) for classification models. 2. model on entire dataset but as can be seen on data I do suspect that league will be important and will be top in predictability power, using boosted tree classification methods and sample features will give a robust model from (exclude date-time or do some feature engineering based on it: like year and group them or some more robust FE will help, otherwise exclude them or use date time for reliability analysis to boost predictions in other words Accelerated failure time (AFT)
– n1tk
Jul 17 at 22:42
• competition should be a feature for the model, not a criteria to split the data Jul 19 at 13:23
• worth to note, you should not include home or away score, since this is not information you will have prior, but at the end of the match (when predicting is no longer necessary) Jul 19 at 13:25

Considering a team like Chelsea has played FA Cup, Champions League, Premier League and other competitions. We need to keep in mind that, other teams would also participate in the same competitions. Sports data from all teams in the competitions would help to identify Chelsea's best win against their toughest competitors that they have faced in FA Cup, Champions League, Premier League.

1. So to answer your first question, you need to take training data for all competitions Chelsea's faced.
2. Coming to second question, what is noise in this model.
To identify noise in a signal we can use statistics, noise would be something that's happened by fluke, in other words Randomness. For better understanding, do have a look at Signal Vs Noise in statistics you tube video.

The signal is the meaningful information that you’re actually trying to detect.
The noise is the random, unwanted variation or fluctuation that interferes with the signal.

1. Coming to your final question What is the most useful approach data science wise?
Most useful approaches are neural networks which outperformed all ML algorithms or building a hyrbid model.
Sports Analytics for Football League Table and Player Performance Prediction: Sports analytics were studied especially team performance prediction and player performance. This paper also provides information on comparative studies of all methods, some of the most promising methods I found are mentioned below.

• Method 1: Dixon, M.J. and Coles, S.G. Modelling Association Football Scores and Inefficiencies in the Football Betting Market

predicting the result of a match was created in 1997 by Dixon and Coles. The model is considered a classic and was able to extract probabilities for the goals scored in a match, following Poisson distribution

• Method 2: Neural Networks
Hucaljuk and Rakipovic utilised Multilayer Perceptron, trained with Back Propagation, equipped with conjugative–gradient algorithms. They concluded that NNs performed better than any other ML technique they used.
• Method 3: Hybrid model
Goddard, in 2005, compared the two methods, i.e. modeling the goals scored vs modeling win–draw–lose match result and concluded that a hybrid model achieves the best prediction performance
• Method 4: Rating concept widely utilised by researchers, most popular ELO Rating
• Method 5:Multiple regression model developed by Oberstone.
He also used F distribution to compare means of multiple samples (i.e. one–way analysis of variance) to investigate which pitch actions differentiate the four best teams from all the others in the league. He managed to achieve outstanding results

#### Research Papers proof

1. Predicting sport outcomes by mere recognition
This paper presents power of recognition in forecasting soccer games. The studies are done on World Cup 2006 and UEFA Euro 2008. Performance measures utilized are ranking rule and odds rule

2. Dolores provides empirical proof that a model can make a good prediction for a match outcome between teams x and y even when the prediction is derived from historical match data that neither x nor y participated in.
While this agrees with past studies in football and other sports, this paper extends the empirical evidence to historical training data that does not just include match results from a single competition but contains results spanning different leagues and divisions from 35 different countries.

3. Principal Component Analysis (PCA), along with ML algorithms (Naive Bayes and Multilayer Perceptron) to predict the Dutch football championship. They achieved an accuracy of almost 55% in their predictions and proved that a hybrid model, combining public data and betting odds could improve accuracy

• Research Paper no. 2 is really good! But, the model argues that even when historical data is not provided for the competing teams providing a capability of providing 'good or better' with historical data. Thank you for these resources! Jul 20 at 6:25
• @pynoob I am glad my answer has been helpful. please consider accepting it by clicking the check-mark. This indicates to the wider community that you've found a solution and gives some reputation to both the answerer and yourself. Jul 20 at 7:48

You need to include all competitions for a simple reason: you'll not have enough data if you do not. (Keep in mind that ML models generelly need large datasets while you only have a couple of matches for a given team in a given year in a given competition if it is not the national league)

In their paper Learning to predict soccer results from relational data with gradient boosted trees the authors found that even when predicting national football leagues (that is, there's no overlap in terms of teams across leagues) it worked better to train their models on the whole dataset instead of training a model per league. Even though this is task and dataset specific, it showcases that in your case restricting the model to a single competition is likely to perform worse.

However, it is then important to include information on the competitions in the dataset because teams might perform differently depending on the type of competition. The trivial approach being to simply include a feature indicating the type of competition but I recommend to provide your model with better engineered features.

For example, you could use competition specific win rates per team, average goals scored per competition or manually assigned importance ratings of each competition. This goes back to the analysis of match importance which has been discussed in football match prediction across different papers. In the paper I linked above you'll find more details on it and different approaches to tackle it. (See section 4.5)

Model-wise the best performance I've seen across a range papers was usually with gradient boosted trees or, in some cases, neural nets. Therefore, I'd suggest to start with boosting (for example use XGBoost) since it's more robust, easier to tune and off-the-shelf solutions are more easily available than in the case of neural nets.

Keep in mind to benchmark your model against a range of baseline models incl. predicting the home team as a winner and predictions based on odds (the latter will be very hard to beat).

• > However, it is then important to include information on the competitions in the dataset because teams might perform differently depending on the type of competition. > Yep, you got part right Jul 17 at 5:54
• Dont know though if I can increase a 'for that competition' bias however GridSearchCv comes with min_weight_fraction_leaf but I found that it provides a smoothed output. Not that its leading to a wrong output (comparing this outpout next day with what has actually happened) however, if I can get any help in this area, I would be welcome. Jul 17 at 6:04
• "but I recommend to provide your model with better engineered features." What can I help provide more? I have a basic model that is used to predict a whole lot of matches. Country, competition, teams bookie odds and the actual scores that have been played in history. If I were to go about it without a prediction method, decision trees is where I would go. Jul 17 at 6:04
• "Model-wise the best performance I've seen across a range papers was usually with gradient boosted trees or, in some cases, neural nets. Therefore, I'd suggest to start with boosting (for example use XGBoost)" Yes, that approach is the next stage if you can call it that. Now, i am getting an output that I can use rightaway. If you can help me build it, I would be grateful for the help. I understand DS is not a code writing service however, I will help in whatever data or models that I can.. Jul 17 at 6:04
• @PyNoob feature engineering is essential in match prediction. 'min_weight_fraction_leaf' will not help with that. Moreover, you only have very few and rather low level features except for the odds but match prediction usually depends on higher level features, such as the ones I listed in my answer. I highly recommend to read the paper I provided to gain an understanding of what features can improve model performance. You can use the data you have to build much more powerful features (i.e. as a starting point you do not need to gather additional data but you need to leverage what you have). Jul 17 at 13:23

Elo rating system is a very useful way to model sport matches by calculating the relative skill levels of different competitors. The difference in Elo ratings between two competitors serves as a predictor of the outcome of a match.

One formula for soccer Elo is: $$Rn = Ro + K × (W - We)$$

• Rn is the new rating
• Ro is the old (pre-match) rating;
• K is the weight constant. K is then adjusted for the goal difference in the game. It is increased by half if a game is won by two goals, by ¾ if a game is won by three goals, and by ¾ + (N-3)/8 if the game is won by four or more goals, where N is the goal difference;
• W is the result of the game (1 for a win, 0.5 for a draw, and 0 for a loss);
• We is the expected result from this formula. We = 1/(10(-dr/400)+ 1) in which dr equals the difference in ratings plus 100 points for a team playing at home.

One option is write a simulator that takes the historical data line-by-line and calculates the Elo scores for each match. After training on all historical data, the resulting Elo scores have predictive value for future games.

• Appreciate the direction however that was not the question. I am sorry if I did not explain it better. The question is on the lines of PCA and if reducing training data to match testing data is useful in this case as it should technically reduce noise. I understand that Decision trees automatically eliminate noise because of its ensemble models and internal voting however, they are also avereraging systems and a bit of spillover effect is visible in such systems if data is not monitored. Sorry that I did not explain this sexplicitly. Jul 13 at 23:21

Lets take an example. The league is Premier League and the teams playing are Chelsea, X, Y and Z (sorry I don't follow football!). So now you have data for all 4 teams for Premier League. Now comes Champions League and the teams playing are Chelsea, Y and Z (X did not get selected for some reason).

Now ask yourself if you should consider data only for Champions League or other league's as well. The answer is yes you should consider data for each and every league, because what you want is predicting win or lose scenario for Chelsea, and Chelsea played against Y and Z in Premier League and both Y and Z are playing in Champions League too! So considering the performance of Chelsea (which is a measure of weather Chelsea will win or lose) you should be wise to include data from other league's as well. In fact you should take all the data of Chelsea in each and every League and against every team you can find and add it to your database.

Now one would be thinking the match against X in Premier League should be considered as noise because X did not qualify for Champions League. I would say no, as predicting if Chelsea will win or not is essentially predicting Chelsea's performance (against any team!). Performance of a team does not change drastically based on a League. If a team is good, it will perform good more or less in every League and the same goes for a bad team. So if Chelsea did good in Premier against X, Y and Z, it will "most probably" do good in Champions against Y and Z too!

So, if Chelsea played against X in Premier League and we want to predict performance of Chelsea in Champions, we should include data of it's match against X. It is valuable info as far as performance of Chelsea is considered.

As far as models are considered, your best bets would be Random Forest, XGBoost and LGBM models (you should also play around with all the various hyperparameters...optuna! wink wink!)

• Keep the data as is and then predict since the data outside the competitions does not make any difference to the performance of the player.
• Try using Random forest since multiple variables like home team, away team, league, home score, and away score are involved, and since it uses ensemble techniques thus provides a more accurate result as compared to other algorithms.
• Which one is appropriate in this case? RandomForestRegressor or RansomForestClassifier ? Jul 13 at 10:41
• I cannot follow the argument that having multiple (independent) variables favors Random Forest (RF) as a model. Can you elaborate on that? And why would you use RFs instead of other ensemble models, such as Gradient Boosted Trees, or a non-ensemble model, such as a Neural Net? Jul 13 at 10:50
• Good question. RandomForest was the example I followed to get to an output hence its familiarity and hence the usage. I should use Gradient Boosted Trees, or a non-ensemble model, such as a Neural Net however, that would be explored as I am limited with my python coding skills. Jul 17 at 7:34