38
votes
Accepted
How to adjust the hyperparameters of MLP classifier to get more perfect performance
If you are using SKlearn, you can use their hyper-parameter optimization tools.
For example, you can use:
GridSearchCV
RandomizedSearchCV
If you use GridSearchCV,...
- 3,608
16
votes
Accepted
How can I Implement Dropout in SciKit-Learn?
To implement this correctly you need to understand exactly how Dropout works, and where to change SciKit-Learn's MLPClassifier class to implement it. I'll start ...
- 671
14
votes
How do I get the feature importace for a MLPClassifier?
The short answer is that there is not a method in scikit-learn to obtain MLP feature importance - you're coming up against the classic problem of interpreting how model weights contribute towards ...
- 1,708
10
votes
Accepted
Why would one crossvalidate the random state number?
I personally think that the general idea of optimising your model with different random seeds is not a good idea. There are many other, more important, aspects of the modelling process that you can ...
- 15.1k
8
votes
Accepted
Is a multi-layer perceptron exactly the same as a simple fully connected neural network?
Yes, a multilayer perceptron is just a collection of interleaved fully connected layers and non-linearities.
The usual non-linearity nowadays is ReLU, but in the past sigmoid and tanh non-linearities ...
- 27.5k
5
votes
How to adjust the hyperparameters of MLP classifier to get more perfect performance
As a complement to the very practical answer of @BrunoGL, I'd like to give a more theoretical answer. I'd like to suggest everyone trying to adjust hyperparameters of a simple Neural Network to read ...
- 2,444
5
votes
Accepted
Why does the MAE still remain, at all?
There can be other reasons related to the model but the most simple explanation is that the data contains contradicting patterns: if the same features correspond to different target values, there is ...
- 25.9k
4
votes
Accepted
What is the difference between multi-layer perceptron and generalized feed forward neural network?
Well you missed the diagram they provided for the GFNN. Here is the diagram from their page:
Clearly you can see what the GFNN does, unlike MLP the inputs are applied to the hidden layers also. While ...
- 803
4
votes
Accepted
Conceptual questions on MLP and Perceptrons
When the data is linearly inseparable, we use MLP. Here what is meant by "data"--is it the response or the input feature that is linearly inseparable?
This means that a linear function of the input ...
- 2,772
4
votes
Accepted
Are weights of a neural network reset between epochs?
An epoch is not a standalone training process, so no, the weights are not reset after an epoch is complete. Epochs are merely used to keep track of how much data has been used to train the network. It'...
- 6,106
3
votes
More layers in NN give worse result
Maybe you are making a mistake, put your code here. But without seeing your code, these are possible points:
Vanishing problem, I don't think you this problem due to having a very shallow network. ...
- 14.2k
3
votes
Accepted
Neural Network Hidden Layer Selection
First try a simple model: The input layer and the output layers dimension are defined by your data / your problem definition. Then train a model without any hidden layer.
See how good it performs. Is ...
- 19.2k
3
votes
Why is the reported loss different from the mean squared error calculated on the train data?
That's because the square loss is defined as 0.5*MSE.
See definition here:
- 2,294
3
votes
Scikit model is not able to predict sequence correctly
I don't use Python so I can't tell you exactly what is going on but I had a quick look at your data:
A few remarks:
it looks like the vast majority of the points are created artificially by ...
- 25.9k
3
votes
Accepted
Understanding computations of Perceptron and Multi-Layer Perceptrons on Geometric level
The two pictures you show illustrate how to interprete one perceptron and a MLP consisting of 3 layers.
Let us discuss the geometry behind one perceptron first, before explaining the image.
We ...
- 1,044
2
votes
How each layer of a neural net is responsible for one feature
... each layer of a neural network is responsible for recognizing one feature of the input data. For example, if we build a neural network that classifies cars, buses, vans and bicycles, a layer will ...
- 14.2k
2
votes
number of neurons for mnist dataset using mlp?
If you train long enough and have "too many" hidden layer units, then you will eventually have over-training. Usually, the goal is to find the smallest number of hidden units that can do reasonably ...
- 301
2
votes
Is the multilayer perceptron only able to accept 1d vector of input data? If yes, why is this so?
You could say every type of neural network gets 1d input data. It's just more convenient to think about 2d-CNNs taking 2d data because the convolution operation is best illustrated by moving squares ...
- 2,381
2
votes
Accepted
MLP conv layers
The term normally used to refer to "MLP conv layers" nowadays is 1x1 convolutions.
1x1 convolutions are normal convolutions, but their kernel size is 1, that is they only act on one position ...
- 27.5k
2
votes
Adding more layers decreases accuracy
The reason is that by adding more layers, you've added more trainable parameter to your model. You have to train it more. You should consider that MNIST data set is ...
- 14.2k
2
votes
Validation loss differs on GPU vs CPU
Since one could see there is no difference in the minmal loss value, this could well be an issue of differnce in floating point precision with respect to CPU and GPU. You should try to cross check ...
2
votes
Why might a neural network consistently underestimate its target?
Consistently underestimating target could be due to the distribution of the target variable. If the target distribution has a negative-skew (i.e., a long tail towards lower values), then the neural ...
- 22.3k
2
votes
Accepted
Interpreting MLP output
This looks like a case of the model outputting the probability of being in category 1. It then is up to you to decide on the cutoff.
You give an example of an output of $(0.43, 0.56, 0.1, 0.8)$. If ...
- 4,244
2
votes
Coding MLP: good practices?
Would my code be faster if I rewrite it with matrices?
Without seeing the code it's impossible to know, but very likely. Also, I would never model single neurons. Too much overhead without any use. ...
- 19.2k
2
votes
Accepted
How to utilize user feedback due to miss-classification when correct class label is unknown?
This can be accomplished by a modification to multi-class cross-entropy.
We are faced with two types of supervision. First type is "data $i$ belongs to class $k$" denoted by $y_{ik}=1$, and second ...
- 9,442
2
votes
IN CIFAR 10 DATASET
The problem here is the input_shape argument you are using, firstly that is the wrong shape and you should only provide an input shape for your first layer.
For ...
- 9,056
2
votes
Accepted
MLP Parameter tuning - gridsearchCV cannot fit?
Your max iteration values are strings.
max_iter': ['200', '1000', '5000', '10000']
Try
...
- 567
2
votes
how to access weights of individual Neurons in the output layers in MLPs?
I'm guessing you want something like this:
model.layers[-1].get_weights()
- 311
2
votes
Accepted
how to access weights of individual Neurons in the output layers in MLPs?
The callback function can be used with model.layers[-1].get_weights() to get weights per iteration.
...
- 231
2
votes
Different hidden layer architectures deliver the same classification results, is that normal?
There are several things I would like to mention :
I do not think that much change on your architecture will impact a lot. Try comparing 10, 20, 50, 100 or more depth. Difference will be most likely ...
- 1,143
Only top scored, non community-wiki answers of a minimum length are eligible