Newest 'softmax' Questions

2

votes

1answer

29 views

Keras: Custom output layer for multiple multi-class classifications

Hello, I’m quite new to machine learning and I want to build my first custom layer in Keras, using Python. I want to use a dataset of 103 dimensions to do classification task. The last fully connected ...

asked Sep 27 at 9:42

meyer_mit_ai

234 bronze badges

0

votes

0answers

10 views

Update function for NN with logistic and sofmax

Can anyone help me confirm my work or find resources on how to come up with the update function for each layer in the Neural Network for multi-class classification problem i.e I am using logistic as ...

machine-learning logistic-regression backpropagation softmax

asked Sep 26 at 9:37

Tyler

1

vote

0answers

15 views

Is the Cross entropy cost function the same as the Cross entropy loss?

Is the Cross entropy cost function defined as $J(\Theta) = -\frac{1}{m}\sum_{i=1}^{m}\sum_{k=1}^{K}y_{k}^{(i)}log(\hat{p}_{k}^{(i)})$ the same as the one implemented in ...

cost-function softmax

asked Sep 13 at 11:03

rober_dinero

192 bronze badges

0

votes

1answer

53 views

"Up or down but not sideways" bimodal time series prediction - what is the best way to model it?

Say I have a time series (e.g. bitcoin price). I want to predict tomorrow's price, specifically tomorrow's % change in price from today. Let's say this is gaussian distributed, with the mean at 0%. If ...

time-series predictive-modeling gaussian finance softmax

asked Sep 2 at 23:59

Cortexelus

1113 bronze badges

0

votes

0answers

23 views

Softmax Derivative

I've been trying to build a neural network from scratch in python over the last few weeks. I've gotten it working with sigmoid, but trying to implement softmax has been killing me, entirely thanks to ...

machine-learning python neural-network softmax

asked Aug 20 at 17:37

Grant Allan

11 bronze badge

0

votes

0answers

20 views

Andrew Ng Deep Learning Gradient Descent of Softmax is just y_hat - y?

At about 8:30 in the video here: https://www.youtube.com/watch?v=ueO_Ph0Pyqk so for the given example with 4 classes and first ground truth y being [0,1,0,0] and y_hat being [0.3,0.2,0.1,0.4] for ...

gradient-descent backpropagation softmax

asked Jul 31 at 20:21

webStudent33

12 bronze badges

0

votes

1answer

25 views

Multiple targets in a classification problem

I have a vector of length $n \gt 4$ which has exactly 4 targets, so for example [0, 0, 0, 1, 0, 1, 0, 1, 1]. I would like to know how I can modify the softmax ...

multiclass-classification softmax multi-output

asked Jul 21 at 5:26

Dennis

1

vote

1answer

65 views

Difference in performance Sigmoid vs. Softmax

For the same Binary Image Classification task, if in the final layer I use 1 node with Sigmoid activation function and ...

image-classification training loss-function softmax sigmoid

asked Jun 28 at 19:20

Eric Cartman

214 bronze badges

0

votes

2answers

18 views

Attention transformation - matrices

Could somebody explain which matrix dimension should be found here - K? and if it is for example 3X3, should I use just 9?

deep-learning transformer matrix softmax

asked Jun 18 at 9:54

Adolf Miszka

1475 bronze badges

0

votes

0answers

9 views

Matrix dimensions issue on calculating Softmax Derivatives

I am trying to get the matrix dimensions right for computing derivative of a two layers network where the last layer is softmax function. For simplicity I am only interested to get derivatives of W2 w....

neural-network backpropagation softmax

asked May 6 at 22:02

ali

1334 bronze badges

0

votes

2answers

33 views

In classification task, is it possible that a truly classified data has a higher loss compared to a miscallisfied data?

Given a classifier using softmax, is it possible that say, for data point a which our model has correctly classified, has higher loss compared to data point ...

classification loss-function softmax

asked Apr 6 at 13:40

m0ss

1033 bronze badges

0

votes

0answers

13 views

A low-degree polynomial alternative for softmax?

I want to train on a low footprint NN which gives decent accuracy on IoTs. I find that the classification using softmax is tedious and very slow. I am looking for a low-degree polynomial alternative ...

deep-learning training research softmax

asked Mar 24 at 5:16

Jaswin

1412 bronze badges

1

vote

0answers

17 views

Convnet with peculiar loss function not learning!

Im using this loss function: ...

neural-network cnn convolutional-neural-network loss-function softmax

asked Mar 23 at 18:36

Cosapocha

112 bronze badges

1

vote

1answer

49 views

Derivative of a custom loss function with the logistic function

I have costum loss function with $\mu ,p, o, u, v$ as variables and $\sigma$ is the logistic function. I need to derive this loss function. Due to multiple variables in the loss function, I need to ...

logistic-regression gradient-descent mathematics softmax

asked Feb 12 at 17:28

Stav Yosef

111 bronze badge

2

votes

0answers

34 views

Using 2 nodes in the output sigmoid activation function for 2 mutually exclusive classes is somehow giving good results than softmax

I know for two mutually exclusive classes softmax is the best activation function in the output layer. However, somehow (2, softmax) and even (1,sigmoid) are giving average results and (2, sigmoid) as ...

deep-learning neural-network classification softmax

asked Feb 9 at 4:54

Tag_DL

212 bronze badges

1

vote

0answers

12 views

Ensemble of different reservoirs (echo state networks)

Suppose I want to do reservoir computing to classify the input to the proper category (e.g. recognizing a handwritten letter). Ideally, after training a single reservoir and testing it, there would be ...

machine-learning neural-network probability ensemble-learning softmax

asked Jan 25 at 2:57

Wouter

1111 bronze badge

3

votes

3answers

535 views

Dot product for similarity in word to vector computation in NLP

In NLP while computing word to vector we try to maximize log(P(o|c)). Where P(o|c) is probability that o is outside word, given that c is center word. Uo is word vector for outside word Vc is word ...

nlp word-embeddings word2vec similarity softmax

asked Jan 14 at 15:55

Vivek Dani

1313 bronze badges

0

votes

0answers

17 views

Dependent Classifiction

Most of the classification problems I have seen assume that the classes are kind of independent. For example, a digit classification uses a softmax output that penalizes the output prediction error ...

classification softmax

asked Nov 18 '20 at 15:39

dsp_guy2020

1

0

votes

0answers

15 views

In multinomial logistic regression, how to explain the softmax outputs properly?

I tried to solve multiclass problem ("cat", "dog", "horse") problem and figured out that the more words in test text, the more difference between classes. I grouped the ...

classification logistic-regression softmax

asked Nov 18 '20 at 10:21

jas_0n

233 bronze badges

0

votes

1answer

124 views

Softmax regression cost function code [closed]

I really do not understand what does this code do M = sparse.coo_matrix(([1]*n, (Y, range(n))), shape=(k,n)).toarray() The code is related to calculating the ...

machine-learning multilabel-classification scipy softmax sparse

asked Oct 19 '20 at 11:15

Mostafa Atallah

111 bronze badge

2

votes

0answers

411 views

Precision-Recall Curve Intuition for Multi-Class Classification Utilizing SoftMax Activation [closed]

I am running a CNN image multi-class classification model with Keras/Tensorflow and have established about a 90% overall accuracy with my best model trial. I have 10 unique classes I am trying to ...

python classification multiclass-classification softmax

asked Oct 16 '20 at 18:37

Coldchain9

1592 bronze badges

1

vote

0answers

123 views

Valid approach? LogSoftmax during training, Softmax during inference

I am training a classifier assigning one of four possible classes to each frame in a preprocessed audio stream using pytorch. I am using cross-entropy loss as the loss function for training. It is ...

pytorch softmax

asked Aug 20 '20 at 15:05

Scipio

133 bronze badges

1

vote

2answers

80 views

Lower loss always better for Probabilistic loss functions?

I am working on an neural net int Tensorflow that predicts percentages for win, draw, loss for given data of a game. The labels I provide are always {1, 0, 0}, {0, 1, 0} or {0, 0, 1}. After some ...

neural-network training loss-function probability softmax

asked Jul 2 '20 at 13:37

Evator

153 bronze badges

1

vote

3answers

45 views

What if multiple final prediction values for multi-class Neural Network are equal

If, for example, your final prediction for a multi-class problem, say for ["mouse","cat","dog","lion"], is [0.1,0.3,0.3,0.3], should the neural network predict that this data is "cat","dog" or "lion"? ...

machine-learning neural-network softmax

asked May 19 '20 at 13:28

Conor Kirby

111 bronze badge

1

vote

0answers

192 views

Generalized softmax derivative for implementation with any loss function

I am currently taking some deep learning and neural network (NN) courses, and in addition to performing the course work, am implementing my own "toolkit" of NN techniques to better my understanding of ...

machine-learning neural-network backpropagation activation-function softmax

asked Apr 15 '20 at 9:49

Krothagon

112 bronze badges

0

votes

1answer

26 views

When can you reorder log operations?

For example, you can reorder a softmax + nl (negative likelihood) to log_softmax + nll (negative log-likelihood) Essentially changing log(softmax(x)) to softmax(log(x)) However, what are the ...

logistic-regression mathematics softmax

asked Mar 12 '20 at 22:37

katiex7

1996 bronze badges

2

votes

1answer

62 views

Problem with chain rule in softmax layer when differentiated separately

I have some problems with backpropagation in softmax output layer. I know how it should work but if I try to apply the chain rule in the classical way, I get different results compared to when Softmax ...

neural-network backpropagation softmax

asked Dec 13 '19 at 7:39

Display name

1531 silver badge4 bronze badges

1

vote

0answers

20 views

Can someone please explain Lovasz softmax loss? as its a bit difficult to understand why it works well from the original paper [duplicate]

Lovasz Softmax is used a lot these days for segmentation problem and the original paper is really bad at explaining why it works.

loss-function softmax

asked Dec 5 '19 at 11:53

Prativa Das

111 bronze badge

1

vote

0answers

62 views

Balance two crossentropy losses with different number of neurons

I have a model with a few outputs, each output with shape: Shape: (batch_size, labels_1) -> softmax -> ...

classification loss-function softmax imbalanced-learn

asked Dec 4 '19 at 13:15

Daniel Möller

1089 bronze badges

0

votes

0answers

25 views

Keras Softmax is Hardmaxing for some reason

I am new to Keras and am bit confused at the moment: ...

keras softmax

asked Dec 3 '19 at 2:27

Tank

2572 silver badges7 bronze badges

-1

votes

1answer

1k views

Softmax gives output vector whose sum is greater than 1 in Pytorch

I am a newbie to PyTorch. I was trying out the following network architecture to train a multi-class classifier. I used Softmax at the output layer and cross entropy as the loss function. However, the ...

neural-network multiclass-classification pytorch softmax

asked Nov 14 '19 at 12:40

Ms_curious_potato

12 bronze badges

1

vote

0answers

64 views

Multi-label classification with missing labels

I have a neural network that generates a vector that represents the class probabilities. Since it is a multilabel classification problem, I'm supposed to train the network using sigmoid + binary cross-...

multiclass-classification multilabel-classification softmax

asked Oct 26 '19 at 19:31

robowolf

111 bronze badge

0

votes

2answers

404 views

How to calculate Temperature variable in softmax(boltzmann) exploration

Hi I am developing a reinforcement learning agent for a continous state/discrete action space. I am trying to use boltmzann/softmax exploration as action selection strategy. My action space is of size ...

deep-learning reinforcement-learning dqn ai softmax

asked Sep 27 '19 at 14:51

cvg

4956 silver badges16 bronze badges

1

vote

0answers

39 views

temperature variable in boltzmmann-exploration in reinforcement learning

I have been using epsilon greedy action selection strategy and recently have come across boltzmann(softmax) action selection strategy. One thing I am not clear about boltzmann exploration is the ...

deep-learning reinforcement-learning ai softmax deepmind

asked Sep 26 '19 at 8:01

cvg

4956 silver badges16 bronze badges

1

vote

0answers

190 views

boltzmann-exploration(softmax exploration) in reinforcement learning

I have started learning reinforcement learning and as a part of it I am exploring the action selection strategies available. I am comparing epsilon-greedy vs boltzmann exploration(softmax exploration)....

reinforcement-learning softmax

asked Sep 25 '19 at 15:29

cvg

4956 silver badges16 bronze badges

20

votes

4answers

9k views

Gumbel-Softmax trick vs Softmax with temperature

From what I understand, the Gumbel-Softmax trick is a technique that enables us to sample discrete random variables, in a way that is differentiable (and therefore suited for end-to-end deep learning)....

neural-network deep-learning attention-mechanism softmax

asked Aug 29 '19 at 10:30

4-bit

3012 silver badges3 bronze badges

3

votes

1answer

377 views

Softmax activation predictions not summing to 1

I am a beginner with rnns, consider this sample code ...

keras tensorflow rnn softmax

asked Aug 24 '19 at 7:40

user329387

311 bronze badge

1

vote

1answer

710 views

Backpropagation with log likelihood cost function and softmax activation

In the online book on neural networks by Michael Nielsen, in chapter 3, he introduces a new cost function called as log-likelihood function defined as below $$ C = -ln(a_y^L) $$ Suppose we have 10 ...

neural-network backpropagation softmax

asked Aug 18 '19 at 13:18

Nagabhushan S N

5741 gold badge5 silver badges19 bronze badges

3

votes

0answers

187 views

Can I turn any binary classification algorithms into multiclass algorithms using softmax and cross-entropy loss?

Softmax + cross-entropy loss for multiclass classification is used in ML algorithms such as softmax regression and (last layer of) neural networks. I wonder if this method could turn any binary ...

machine-learning multiclass-classification softmax

asked Jul 30 '19 at 6:22

CouldntLoginToMyPreviousAcc

1313 bronze badges

3

votes

3answers

3k views

Do I need to standardize my one hot encoded labels?

I'm trying to do a simple softmax regression where I have features (2 columns) and a one hot encoded vector of labels (two categories: left = 1 and Right = 0). Do I need to standardize just the ...

machine-learning softmax

asked Jul 26 '19 at 13:39

José Lucas Araújo dos Santos

311 silver badge2 bronze badges

0

votes

2answers

100 views

Is the Cross Entropy Loss important at all, because at Backpropagation only the Softmax probability and the one hot vector are relevant?

Is the Cross Entropy Loss (CEL) important at all, because at Backpropagation (BP) only the Softmax (SM) probability and the one hot vector are relevant? When applying BP, the derivative of CEL is the ...

deep-learning loss-function backpropagation softmax

asked Jul 20 '19 at 6:07

clemens_m_b

1

3

votes

1answer

4k views

Pytorch doing a cross entropy loss when the predictions already have probabilities

So, normally categorical cross-entropy could be applied using a cross-entropy loss function in PyTorch or by combing a logsoftmax with the negative log likelyhood function such as follows: ...

neural-network loss-function probability pytorch softmax

asked Jul 18 '19 at 21:56

user3023715

1831 silver badge5 bronze badges

2

votes

1answer

426 views

Why use different variations of Softmax in training and validation for neural networks with Pytorch?

Specifically, I'm working on a modeling project, and I see someone else's code that looks like ...

neural-network deep-learning pytorch softmax

asked Jul 10 '19 at 21:11

Anon

1234 bronze badges

2

votes

2answers

565 views

Why do we use a softmax activation function in Convolutional Autoencoders?

I have been working on an image segmentation project where I have created a convolutional autoencoder. I saw this image and implemented it using Keras. At the output layer, the author has used the ...

convolution image-recognition autoencoder activation-function softmax

asked Jun 15 '19 at 5:37

Shubham Panchal

1,8775 silver badges18 bronze badges

1

vote

1answer

86 views

Softmax function result for already normalized probabilities

Isn't the aim of softmax function normalizing the probabilities such that they all sum to 1? So when we apply this method to the already normalized numbers, it would change them. what do these new ...

cnn normalization softmax

asked May 22 '19 at 22:27

Fatemeh Asgarinejad

1,1344 silver badges17 bronze badges

1

vote

1answer

35 views

Does the score output of a classification model has a global meaning?

The scores-output layer contains the class scores that the model generated for the current sample and it is passed thru the softmax layer to get the final output in the form of a probabilities vector. ...

classification object-detection softmax

asked May 20 '19 at 12:33

Mark.F

1,9602 gold badges12 silver badges24 bronze badges

3

votes

1answer

495 views

Multiclass Classification with Decision Trees: Why do we calculate a score and apply softmax?

I'm trying to figure out why when using decision trees for multi class classification it is common to calculate a score and apply softmax, instead of just taking the averages of the terminal nodes ...

decision-trees xgboost softmax

asked Sep 27 '17 at 1:05

Caleb

1312 bronze badges

65

votes

6answers

128k views

Cross-entropy loss explanation

Suppose I build a neural network for classification. The last layer is a dense layer with Softmax activation. I have five different classes to classify. Suppose for a single training example, the <...

machine-learning neural-network deep-learning softmax

asked Jul 10 '17 at 10:26

enterML

2,8817 gold badges24 silver badges37 bronze badges

Questions tagged [softmax]

Related Tags