New answers tagged loss-function
0
Based on how the gradient descent optimization algorithm works (which you can find extensevily explained in this link), find below the answers to your questions:
Has all the loss surface flat area around the optimal thus causing small gradient updates?
as far as the updated weights approximate the optimal weight values (and this happens on what you call ...
1
There is usually no straightforward interpretation of what cross-entropy means in the context of the given task. In practice, it is more important to follow the trend of how the cross-entropy develops during the training.
The measure comes from the information theory. It says how surprised the model is when it has some belief about the output (a distribution)...
1
This should be possible using a piecewise exponential loss, something like this:
$
f(x) =
\begin{cases}
x^2 & x < 0\\
\lambda x^2 & x \ge 0
\end{cases}
$
with $0 < \lambda < 1$. A $\lambda$ of around 0.02 should roughly give you the scale you want.
0
They are different:
L2 = $\sqrt{\sum_{i=1}^{N}(y_i-y_{i}^{pred})^2}$
MSE = $\frac{\sum_{i=1}^{N}(y_i-y_{i}^{pred})^2}{N}$
There are sum and square root for L2-Norm, but sum and mean for MSE!
We can check it by following code:
import numpy as np
from sklearn.metrics import mean_squared_error
y = np.array(range(10, 20)) # array([10, 11, 12, 13, 14, 15, 16, ...
1
Following on Thomas on the relation between the Bray Curtis distance and the F1 score and the calculation of the first and second-order derivatives. If one defines the Bray Curtis distance between vector X and Vector Y as: $\sum |X_i-Y_i| \over {\sum (X_i+Y_i)}$, than the first derivative to $x$ is $d \over (dx)$ $|x - y| \over {(x + y)}$ = $2y(x - y) \over{\...
0
You should have a list of actual classes, e.g. classes = ['Superman', 'Batman', ...,'Gozilla']. The model outputs per-class logits, but without your dataset interface it's hard to say what your targets is. Since it's a multiclass problem, it should be an integer between 0 and 5. I assume the order of targets and the order of classes in classes list is the ...
0
If you're dealing with classification problem, then model.predict is supposed to give you logits.
outputs = net(images)
_, predicted = torch.max(outputs, 1)
for i in range(num_input):
print(classes[predicted[i]])
if you have only one input then the predicted class would be as following:
classes[predicted[0]]
In your case:
prediction = F.softmax(...
0
The problem is that, in Pytorch, CrossEntropyLoss is more than its name suggests. The documentation says that:
This criterion combines nn.LogSoftmax() and nn.NLLLoss() in one single class.
This should behave like you expect:
Loss = nn.NLLLoss()
y = torch.tensor([0.25, 0.25, 0.29, 0.21]).unsqueeze(0)
y_true = torch.tensor([2])
Loss(torch.log(y), y_true)
...
Top 50 recent answers are included
