Questions tagged [softmax]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1
vote
0answers
13 views

Can someone please explain Lovasz softmax loss? as its a bit difficult to understand why it works well from the original paper [duplicate]

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.
1
vote
0answers
13 views

Balance two crossentropy losses with different number of neurons

I have a model with a few outputs, each output with shape: Shape: (batch_size, labels_1) -> softmax -> ...
0
votes
0answers
11 views

Keras Softmax is Hardmaxing for some reason

I am new to Keras and am bit confused at the moment: ...
0
votes
0answers
11 views

NN SoftMax/CE getting incorrect values when differentiated independently

So I solved it once, but can't recall how I did it... I'll show strictly where my problem is. $$ Z^{O} := (1.57, 1.61) $$ $$ A^{O} := (0.49, 0.51) $$ Where Z is the layer output before applying the ...
-1
votes
1answer
39 views

Softmax gives output vector whose sum is greater than 1 in Pytorch

I am a newbie to PyTorch. I was trying out the following network architecture to train a multi-class classifier. I used Softmax at the output layer and cross entropy as the loss function. However, the ...
1
vote
0answers
19 views

Multi-label classification with missing labels

I have a neural network that generates a vector that represents the class probabilities. Since it is a multilabel classification problem, I'm supposed to train the network using sigmoid + binary cross-...
0
votes
0answers
13 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...
1
vote
0answers
11 views

temperature variable in boltzmmann-exploration in reinforcement learning

I have been using epsilon greedy action selection strategy and recently have come across boltzmann(softmax) action selection strategy. One thing I am not clear about boltzmann exploration is the ...
1
vote
0answers
63 views

boltzmann-exploration(softmax exploration) in reinforcement learning

I have started learning reinforcement learning and as a part of it I am exploring the action selection strategies available. I am comparing epsilon-greedy vs boltzmann exploration(softmax exploration)....
1
vote
1answer
460 views

Gumbel-Softmax trick vs Softmax with temperature

From what I understand, the Gumbel-Softmax trick is a technique that enables us to sample discrete random variables, in a way that is differentiable (and therefore suited for end-to-end deep learning)....
3
votes
1answer
40 views

Softmax activation predictions not summing to 1

I am a beginner with rnns, consider this sample code ...
1
vote
1answer
65 views

Backpropagation with log likelihood cost function and softmax activation

In the online book on neural networks by Michael Nielsen, in chapter 3, he introduces a new cost function called as log-likelihood function defined as below $$ C = -ln(a_y^L) $$ Suppose we have 10 ...
0
votes
0answers
28 views

How exactly should i use softmax activation in last layer of a Neural Network

I am developing a digit classifer with MNIST Dataset. I have read that for classification problems softmax activation function is used, as it maps last layer neuron's outputs into probabilities. ...
2
votes
0answers
37 views

Can I turn any binary classification algorithms into multiclass algorithms using softmax and cross-entropy loss?

Softmax + cross-entropy loss for multiclass classification is used in ML algorithms such as softmax regression and (last layer of) neural networks. I wonder if this method could turn any binary ...
0
votes
2answers
171 views

Do I need to standardize my one hot encoded labels?

I'm trying to do a simple softmax regression where I have features (2 columns) and a one hot encoded vector of labels (two categories: left = 1 and Right = 0). Do I need to standardize just the ...
0
votes
2answers
24 views

Is the Cross Entropy Loss important at all, because at Backpropagation only the Softmax probability and the one hot vector are relevant?

Is the Cross Entropy Loss (CEL) important at all, because at Backpropagation (BP) only the Softmax (SM) probability and the one hot vector are relevant? When applying BP, the derivative of CEL is the ...
1
vote
1answer
746 views

Pytorch doing a cross entropy loss when the predictions already have probabilities

So, normally categorical cross-entropy could be applied using a cross-entropy loss function in PyTorch or by combing a logsoftmax with the negative log likelyhood function such as follows: ...
0
votes
0answers
26 views

Thresholding in intermediate layer using Gumbel Softmax

In a neural network, for an intermediate layer, I need to threshold the output. The output of each neuron in the layer is a real value, but I need to binarize it (to 0 or 1). But with hard ...
2
votes
1answer
54 views

Why use different variations of Softmax in training and validation for neural networks with Pytorch?

Specifically, I'm working on a modeling project, and I see someone else's code that looks like ...
1
vote
2answers
118 views

Why do we use a softmax activation function in Convolutional Autoencoders?

I have been working on an image segmentation project where I have created a convolutional autoencoder. I saw this image and implemented it using Keras. At the output layer, the author has used the ...
0
votes
0answers
122 views

Why is the last layer of a Tensorflow hub model not fed directly to softmax?

I was checking out this example using BERT model for movie review sentiment analysis, and a few others on how to use Tensorflow hub models. A common pattern that I notice is the output from the last ...
1
vote
1answer
47 views

Softmax function result for already normalized probabilities

Isn't the aim of softmax function normalizing the probabilities such that they all sum to 1? So when we apply this method to the already normalized numbers, it would change them. what do these new ...
1
vote
1answer
28 views

Does the score output of a classification model has a global meaning?

The scores-output layer contains the class scores that the model generated for the current sample and it is passed thru the softmax layer to get the final output in the form of a probabilities vector. ...