New answers tagged cross-validation
0
1.) If your aim is to find the most relevant features the first thing you should do is feature engineering using Domain Knowledge. It is the most effective feature selection technique out there and in done properly, does not require any extra feature selection techniques.
2.) After you do the above step, if you want to get a measure of "importance" ...
0
This is an interesting one and there is not a "one-fits-all" answer to it. If I break down your question into two major parts, I would say:
Choosing important variables
Domain expert: It is always helpful to have an idea from the domain expert on what variables matter the most, especially in your case that you have 1000 variables to choose from. ...
1
This is about distributed computing: let's say that you have 100 tasks and 10 cores available. You parallelize your tasks so that each core processes 10 of them. Now let's imagine that the task involves some subtasks and internally tries to use all the cores available: at the two levels of parallelization the processes compete for the cores, causing a loss ...
0
As stated in the earlier comments, preprocessing both train and test set at the same time causes serious generalization error in the real life applications. So, I totally agree with you at this point.
When it comes to scaler issue, the first thing that I came up with is that you can evaluate the importances of scaled features over the target value. Then, you ...
0
Ok first things first, I do not think it's a good idea to concatenate train and test set for anything and you are right in stating the problem of data leakage.
Now as to getting different results when using scaler in cv is expected. This is because, for every iteration of cv, the data set changes. For example if cv = 3, then for the first iteration, it will ...
1
You get to use the entire data you have as part the training process (so the inner CV would essentially get to see all the data at some point).
The model performance estimate you get could be more stable (in the sense that it is not based on a single run using the test data, but on multiple runs.
You've covered the main benefits. However, it is important to ...
Top 50 recent answers are included
