I'm currently working on a Kaggle Competition wich objective is to predict probabilities of an ID belonging to each class. There are 4 posible classes.
The data is tabular and because it's a Kaggle 'just for fun' Competition, the features are already clean and the data is semi-ready to train the model.
After standarizing (MinMaxScaler()) the data and splitting into train, test and validation, the data is ready to train.
Ok, till there everything it's fine. We have our train data (X). For the target variable (Y) we use One Hot Encoder, so then we have a
Nx4 matrix for Y (N: number of samples; 4 number of clases). I choose to encode the (Y) variable with One Hot Encoder because I need as output a matrix of shape
Nx4 with the predicted probabilities for each sample of belonging to each class.
First approach was a Neural Network because with an output layer with softmax activation function it would give as output exactly what I wanted to. I tried with different architectures for the Neural Net but I can't make it work, the training and validation accuracy are divergent, so it doesn't generalize good.
After trying different Neural Nets, I tried with a Random Forest Classifier with Multi-Output, but in this case the predicted probabilities that it return are all for one-vs-rest models.
There is any way to solve a multi label classification with shallow machine learning algorithms so the output is like a softmax output layer?
Thank you for reading till here, any help will be very welcome.