Use all your data, throw a bunch of ensemble ML (probably just random forest) at it, and pick a good model. Usually, that works exceptionally well.
How much data should you use?
Ideally, you should always be separating your data into Train, Test, and Validation. Due to small dataset sizes, this isn't always possible, but it is still important to prevent overfitting. You can use all your training and testing data for feature selection and that shouldn't introduce any significant biases to your ML model. Your validation set should only be used to approximate the error once you are entirely done training your model.
Simple ML Solutions:
Use some kind of random forest or gradient boosting model on all your training/testing data. These models are designed for high performance in high dimensional data and by checking feature importances/permutation importances/mean decrease in accuracy you will know which features are/aren't important. It is common to use these kinds of models in the biological space where there are millions of features and only a thousand or so responses. Despite the disparity, I've still reached high accuracy in these situtations.
Simple Statistical Solutions
Measure the correlation of each feature and keep only those features that have an absolute correlation above/below a certain amount. You can use a few statistical tests to filter out features that are not significant. Here are a few examples:
- Pearson Correlation
- Lasso Regression (technically an ML algorithm)
Pitfalls of the Above
Each of the above uses some kind of assumption to figure out which features you should select for final model training. Sometimes you don't actually need to pick a subset of features ie when you use RF. Additionally, the statistical tests often miss abnormal types of correlations or miss multidimensional relationships. Feature selection is really a case-by-case decision that no-one can give you a definitive answer on without seeing the actual data you use.