Gradient Descent is an algorithm for finding the minimum of a function. It iteratively calculates partial derivatives (gradients) of the function and descends in steps proportional to those partial derivatives. One major application of Gradient Descent is fitting a parameterized model to a set of data: the function to be minimized is an error function for the model.

355 questions
Filter by
Sorted by
Tagged with
51 views

### How to find slope of curve at certain points

how to find slope at certain points circled in blue in below curve ? Are these below 2 approaches valid ? though they give different results . How to automatically find the points where the slope ...
24 views

### Multivariable linear gradient descent resulting in inf

I am trying to implement a multivariable gradient descent algorithm, it seems to start working fine, and works on smaller datasets, but applying it to larger datasets the variables overflow and cause ...
42 views

### Why do we only care about convex functions when doing Gradient Descent/SGD?

I mean I know why we specifically care about convex functions: it's because their local minimum are also global, and so you just have to "follow a path which goes down" to find the minima of ...
68 views

### gradient descent for non convex function like $-x^2$

I know how to calculate gradient descent for a convex function where there is only one global minima. Also, I know methods to handle cases where the function is a non-convex function. What is really ...
110 views

### Linear Regression in Python using gradient descent

I am trying to implement a simple multivariate linear regression model without using any inbuilt machine libraries. So far, I have been able to get a root mean squared error for training about $2.93$ ...
38 views

### Gradient descent does not converge in some runs and converges in other runs in the following simple Keras network

When training a simple Keras NN (1 input, 1 level with 1 unit for a regression task) during some runs I get big constant loss that does not change in 80 batches. During other runs it decreases. What ...
91 views

### Getting NN weights for every batch / epoch from Keras model

I am trying to get weights for every batch / epoch from Keras model after it is trained. To do so I use callback to make model save weights during training. Yet after model is trained it looks like I ...
23 views

### Why use gradient descent on Deep Nets / RNNs when cost function is not convex?

Why do we use gradient descent on very non-convex loss functions such as in Deep nets / RNNs rather than a heuristic search (genetic algorithms, simulated annealing, etc)?
138 views

### What is the significance of underflow during parameter update using stochastic gradient descent?

Background I am using scikit-learn's MLPRegressor to learn a model with the following arguments: ...
29 views

The constrained optimization problem in SVM is given by min 1/2 ||w||^2 s.t y(i)(w^T x(i) + b >= 1 for all i Now converting this to an unconstrained optimization problem gives the lagriangian L as ...
38 views

### Why standard distribution for ML [closed]

Data normalization: It ensures that each input (each pixel value, in this case) comes from a standard distribution. This standardization makes our model train and reach a minimum error, faster! my ...
140 views

I was reading about descending gradient. How does the descending gradient know what weights to adjust? Does it adjust to all network weights at the same time? Does each weight have an associated error?...
34 views

### Neural Network Loss Function - Mean Square Error: questions about what 'n' signifies

I'm very new to neural networks and have recently learnt about the loss functions used with neural networks. This question is in regards to the mean square error metric, defined as (from the textbook ...
44 views

### Understanding the concept vanishing gradient and exploding gradient problem in terms of training data

I'm trying to figure out the essence of the concepts "vanishing gradient and exploding gradient problem" in terms of real-world input-output training examples instead of in terms of the properties of ...
13 views

30 views

### Finding a vector that minimize the MSE of its linear combination

I have been doing a COVID-19 related project. Here is the question: N = vector of daily new infected cases D = vector of daily deaths E[D] = estimation of daily deaths N is a n-dimensional vector, n ...
83 views

### Is it valid to use numpy.gradient to find slope of line as well as slope of curve at any point?

what is the difference between slope of the line and slope of the curve? Is it valid to use numpy.gradient to find the slope of the line and slope of the curve at ...