Questions tagged [derivation]
The derivation tag has no usage guidance.
18
questions
0
votes
1
answer
31
views
Are some weight gradients equal?
I want to create a 3 layers neural network from scratch to perform linear regression.
The first and the second layer have 2 neurons, and the last layer has one neuron.
Feature vector x is divided into ...
- 148
0
votes
0
answers
9
views
Derivative of a KernelRidge regression model based on Coulomb Matrix descriptor
I am trying to take analytical derivatives of a KernelRidge regression model that takes as input a Coulomb Matrix descriptor. A Coulomb Matrix is a way of representing a molecular structure basically ...
- 1
0
votes
0
answers
16
views
How does backpropagation through accuracy work?
I'm using a specific constraint on my predicted logits and adding it to the loss.
In a nutshell, this constraint tries to minimize cross-overlap between the channels of my predictions. I'm using ...
- 13
0
votes
0
answers
88
views
calculating derivative of bias in backpropagation
Looking at the algorithm in wikipedia, we can implement backpropagation by calculating:
$$\delta^{L}=\left(f^{L}\right)'\cdot\nabla_{a^{L}}C$$
(where I treat $\left(f^{L}\right)'$ as an $n\times n$ ...
- 101
1
vote
0
answers
43
views
How to compute backpropagation gradient according chain rule for using vector/matrix differential?
I have some problems for computing derivative for sum of squares error in backprop neural network.
For example, we have a neural network as in picture. For drawing simplicity, i've dropped the sample ...
1
vote
0
answers
183
views
Adding a group specific penalty to binary cross-entropy
I want to implement a custom Keras loss function that consists of plain binary cross-entropy plus a penalty that increases the loss for false negatives from one class (each observation can belong to ...
- 11
0
votes
1
answer
88
views
Loss function for points inside polygon
I am trying to optimize some parameters that used to transform 2d points from a place to another (you may think of that as rotation & translation parameter for simplicity)
The parameters are ...
- 101
1
vote
1
answer
111
views
SVM - Making sense of distance derivation
I am studying the math behind SVM.
The following question is about a small but important detail during the SVM derivation.
The question
Why the distance between the hyperplane $w*x+b=0$ and data ...
- 21
2
votes
0
answers
64
views
Deriving vectorized form of linear regression
We first have the weights of a D dimensional vector $w$ and a D dimensional predictor vector $x$, which are all indexed by $j$. There are $N$ observations, all D dimensional. $t$ is our targets, i.e, ...
- 143
-1
votes
1
answer
190
views
Why is it valid to remove a constant factor from the derivative of an error function?
I was reading the book 'Make your own neural network' by Tariq Rashid. In his book, he said:
(Note - He's talking about normal feed forward neural networks)
The $t_k$ is the target value at node $k$, ...
1
vote
1
answer
86
views
How is this score function estimator derived?
In this paper they have this equation, where they use the score function estimator, to estimate the gradient of an expectation. How did they derive this?
- 13
3
votes
1
answer
6k
views
Is it valid to use numpy.gradient to find slope of line as well as slope of curve at any point?
what is the difference between slope of the line and slope of the curve? Is it valid to use numpy.gradient to find the slope of the line and slope of the curve at ...
- 1,361
2
votes
1
answer
196
views
1st order Taylor Series derivative calculation for autoregressive model
I wrote a blog post where I calculated the Taylor Series of an autoregressive function. It is not strictly the Taylor Series, but some variant (I guess). I'm mostly concerned about whether the ...
- 121
1
vote
1
answer
752
views
Derivative of Loss wrt bias term
I read this and have an ambiguity.
I try to understand well how to calculate the derivative of Loss w.r.t to bias.
In this question, we have this definition:
...
3
votes
1
answer
219
views
Maximum Entropy Policy Gradient Derivation
I am reading through the paper on Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review by Sergey Levine. I am having a difficulty in understanding this part of the ...
1
vote
0
answers
18
views
back propagation through time derivation issue
I read several posts about BPTT for RNN, but I am actually a bit confused about one step in the derivation. Given
$$h_t=f(b+Wh_{t-1}+Ux_t)$$
when we compute $\frac{\partial h_t}{\partial W}$, does ...
- 151
2
votes
1
answer
86
views
Doubt in Derivation of Backpropagation
I was going through the derivation of backpropagation algorithm provided in this document (adding just for reference). I have doubt at one specific point in this derivation. The derivation goes as ...
- 175
1
vote
1
answer
19
views
A Derivation in Combinatory Categorial Grammer
I am reading about CCG on page 23 of Speech and Language processing. There is a derivation as follows:
(VP/PP)/NP , VP\((VP/PP)/NP) => VP?
Can anyone example ...
- 153