I guess what you are looking for is sklearn.model_selection.GridSearchCV() or a similar function regarding the type of "search" (grid, random, ...) you would like to conduct. This function does hyperparameter tuning and uses cross validation when doing so and the cv parameter will allow you to specify the number of folds.
Decision Tree is very useful if you want to be able to explain where your result comes from you can often print the tree and see how your model came to this answer.
Random Forest can also provide such information, but you'll have to browse all trees and make some "stats" into them, which is not as easy. But Random Forest often give better results ...
By looking at your datasets, it resembles a multivariate time series problem , not sure why you are opting for Random Forest classifier ?
I would suggest you to start implementing simple statistical
algorithm first here is the link
Then explore on complex Deep Learning algorithm based on the data
set size , please find the referenced link
To have a probability of 1 in a RF, it means that your algorithm can construct a leaf containing only positive sample. Since it doesn't, this means that your features are not explaining the variance of the output or that your algorithm is under-fitted.
I suggest that you try optimize the hyper-parameters of your RF by using cross-validation and use some ...
If the number of points is constant in your array you can flatten your array and use each element as a feature in your RF. I worked on a similar problem (If I understood your problem correctly) where I predict the return of a stock based on his return on a given window of a fixed number of days and I have used the RF this way and it performs pretty well.
I think recall and FPR are calculated in scikit-learn using a threshold of 0.5. On the other hand ROC AUC is transparent to model threshold. I encourage you to explore thresholder in scikit-lego to inspect in this direction.
An example of AUC = 1 but bad FPR would be if you use 0.5 as a threshold, you model splits your samples perfectly but the positive ones ...
Welcome to the community!
There are points coming to my mind:
Check the amount of data in each block and then their distribution. First experience might be due to the lack of enough data in reserved block (i.e. you literally just trained but did not validated resulting to a complete overfitting) or having a totally different distribution (i.e. the ...
I think what you are describing would be called anomaly detection. I suggest trying a different approach. There are several standard solutions to deal with this topic, here are a few.
Main problems to address are:
Setting a good threshold to balance false alarms with missed
events. Selecting the model will influence this setting (see below for 2 typical ...
If you can find a column that has a value to select on, you can use stratify in the train_test_split function. Stratify will try to select an equal number of cases of each value, similar to what you are using. You won't capture all of them, just an equal sample of value vs non-value, but this would be a better approach than forcing a non-random sampling on ...