I'm currently working on a project where my database has about 120 patterns with 39 columns and I am trying to build a predictive neural network out of it. It's regression task.
I was trying to get the best predictors ( alone or combinations of'em) in a simple network( 3 neurons only) to then use cross validation to better tune the model. Problem are 1) the powerset of it is huge and my computer can't even handle generating the whole subset for simple fitting 2) only 3 neurons are already giving pretty poor results (r2<0)
Does somebody know a method or could please recommend a reading on choosing predictors for neural networks ?
Setup: windows 10, using MLPRegressor from scikitleran with hyperparameters Hidden layer sizes = 3, max_iter= 5000, solver='sgd'