4
Considering a team like Chelsea has played FA Cup, Champions League, Premier League and other competitions. We need to keep in mind that, other teams would also participate in the same competitions. Sports data from all teams in the competitions would help to identify Chelsea's best win against their toughest competitors that they have faced in FA Cup, ...
3
You are currently using the fit_transform method on both your training dataset as well as your test set. This is incorrect since you should not fit the model on your test set as (depending on the model used) this would be overfitting, and it can give issues with dataset shapes when creating new columns based on the values in the data (count vectorizer, ...
2
Yes it is wrong to set shuffle=True.
By shuffling the data you allow your model to learn properties of the data distribution that might appear only in the test time periods.
For example, if you have a trend in the data, shuffling will 'help' you handle it.
In a real-time scenario, you'll never have access to those properties of the distribution.
1
You need to include all competitions for a simple reason: you'll not have enough data if you do not. (Keep in mind that ML models generelly need large datasets while you only have a couple of matches for a given team in a given year in a given competition if it is not the national league)
In their paper Learning to predict soccer results from relational data ...
1
how can I get from scikit learn BOTH the result and the probability?
You can simply run both:
The results will always be consistent because there is no randomness involved at the prediction stage, only at training stage.
The computations required for predicting are not intensive, so I don't think there can be any major efficiency issue running it twice.
...
1
Instead of GridSearchCV you should try Optuna. It is much faster than GridSearchCV.
But apart from that, coming to your question, there is no best value for a hyperparameter per se! Period! It depends on what kind of data you have. What hyperparameter value works for one dataset might not work for another dataset.
Also another point to keep in mind, there ...
1
In general there's no way to know the best values to try for a parameter. The only thing one can do is to try many possible values, but:
this mathematically requires more computing time (see this question about how GridSearchCV works)
there is a risk of overfitting the parameters, i.e. selecting a value which is optimal by chance on the validation set.
1
Try Optuna which is relatively faster than GridSearchCV. Also n_jobs = -1 further reduces time. Another point is to tune parameters that matter. Not all parameters will give you maximum improvement in results. Read this blog for further info:
https://blog.dataiku.com/narrowing-the-search-which-hyperparameters-really-matter
1
To extend my comment:
As I mentioned you can set the parameter n_jobs to -1 or instead using RandomizedGridSearch (which also receives n_jobs parameter)
Regarding to the parameter grid, I always select my grid so that the default values are included and from there, some values less and greater than the default (for continuous parameters) and the same logic ...
1
If your training data is large enough, the model will have enough information to deal with chance through the statistics in the data. For example maybe a great shot is successful 80% of the time, so if there are 10 instances of great shot in the data there should be around 8 of them successful. In other words, the model will use the distribution of the data ...
1
I ran into a similar issue. Even with small tree sizes, I got a file of hundreds of megabytes.
Check if you've set oob_score=True.
For large training datasets this can result in a large matrix in oob_decision_function_. I kept the oob_score_, but deleted this matrix. Alternatively, you can set it to False.
Only top voted, non community-wiki answers of a minimum length are eligible
