New answers tagged

4 votes
Accepted

AutoDiff on different operations?

I will address your question in a roundabout manner, but you will see why. We can compute a gradient on any convolutional layer, no matter the dimension, because convolution is defined in a similar ...
AndrewJaeyoung's user avatar
7 votes

How does a Neural Net handle an unseen class for a Categorical Feature?

Typically feature X would be presented to the NN via one hot encoding. So we have three (boolean) indicator variables, denoting the presence of A, B, or C. If the problem domain admits of D or E ...
J_H's user avatar
  • 1,045
0 votes

Connecting Flatten layer to Dense layer

I tried this code and it worked fine without any errors : ...
4A4a's user avatar
  • 1
1 vote

What's wrong with my implementation of an MLP?

The issue with your implementation seems to be in the calculation of the gradient for the activation of the hidden layers in the backward propagation step of your neural network. ...
Pluviophile's user avatar
  • 3,848
5 votes
Accepted

Does using different optimizer change the loss landscape

No, the optimizer does not change the loss landscape. The loss landscape consists of the possible parameters of your model and the incured loss of a model which has these parameters. We could for ...
picky_porpoise's user avatar
0 votes
Accepted

How do I ensure final output shape matches input shape for a semantic segmentation task?

The issue comes from the fact that a max pooling operation is applied to downsample at each level of the U-Net. When the your input is not divisible by two the resulting array after max pooling will ...
Oxbowerce's user avatar
  • 7,507
0 votes
Accepted

Why there is no exact picture of softmax activation function?

I'm late for my own party... so in short it is possible to plot after the implementation of the SoftMax activation function, however it could be varied because: A softmax function is a very different ...
Mario's user avatar
  • 400
1 vote

Creating a custom loss function for an image classification model where the label matters

I think it will be very difficult to solve this with the cross-entropy, since this loss only looks at the class of your current prediction and never at any other class (so you cannot really consider '...
picky_porpoise's user avatar
0 votes
Accepted

Pytorch backward error

When you index into a tensor, you are actually creating a new data object. The new data object has a computational graph linking it to the weights tensor via the select operation, but they are not the ...
Karl's user avatar
  • 646
0 votes

Square Root Regularization and High Loss

Instead of square root, ln(1+abs(p)) has a similar curve but avoids the gradient issue near 0. I am considering using this loss function in a regression problem ...
Adriel Jr's user avatar
  • 101
1 vote

What is the best way to train a neural network with a variable number of inputs?

I would just set your missing values in the input layer to zero, or equivalently, apply your binary vector as a mask to the input layer. This would be in effect the same as applying dropout to your ...
babelproofreader's user avatar

Top 50 recent answers are included