Gradient Descent is an algorithm for finding the minimum of a function. It iteratively calculates partial derivatives (gradients) of the function and descends in steps proportional to those partial derivatives. One major application of Gradient Descent is fitting a parameterized model to a set of data: the function to be minimized is an error function for the model.

355 questions
Filter by
Sorted by
Tagged with
51 views

### Will stochastic gradient descent converge for multivariate linear regression

I am trying to figure out if stochastic gradient descent for a multivariate linear regression will converge (assuming there is no mini-batching, i.e., the batch size is 1). My guess is yes, based on ...
52 views

### Why Gaussian mixture model uses Expectation maximization instead of Gradient descent?

Why Gaussian mixture model uses Expectation maximization instead of Gradient descent? What other models uses Expectation maximization to find best optimal parameters instead of using gradient descent?
6 views

### what are Latent model and Latent Varibles?

what are latent models and why they use Expectation maximization instead of gradient descent to optmize the parameters ? How to interpret and understand whether a hidden varible is present in model or ...
106 views

14 views

50 views

### Transposed Convolution without using Python built-in functions

Amateur here: How can we write a 2D transposed convolution (aka deconvolution) using the steepest descent method given the following restrictions: cannot use any Python built-in functions cannot ...
31 views

I was reading this nice tutorial about Pytorch's basics: https://pytorch.org/tutorials/beginner/pytorch_with_examples.html In the first example (pure Numpy), the author starts the backward phase by ...
188 views

### grad-cam implementation on mobilenet SSD network

Below is a gradcam implementation for a standard image classifier : ...
29 views

### Does severe multicollinearity affect solving linear regression by gradient descent?

Since OLS may fail when there is severe/near perfect multicollinearity, how would gradient descent perform in such a scenario? Does it converge at the minima? (My guess is, Cost function of linear ...
53 views

### Tensorflow - Manually decay Adam optimizer

I've been experimenting with reinforcement learning and using the train_on_batch method of tf.keras.models.Model to update the ...
40 views

### Projected gradient descent in keras

I am currently working on a project and I need to do project gradient descent instead of vanilla gradient descent on a network. I am unsure if current deep learning frameworks have that functionality. ...
231 views

### What is the best way to find minima in Logistic regression?

In the Andrew NG's tutorial on Machine Learning, he takes the first derivative of the error function then takes small steps in direction of the derivative to find minima. (Gradient Descent basically) ...
41 views

### Should bias updates be porportional to overfitting?

According to questions on the internet, the bias is a learnable parameter, and there are different solutions to updating it, but I failed to find a concise methodology of correctly updating biases ...