8k views

### local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
11k views

### How to plot cost versus number of iterations in scikit learn?

One of the recommendations in the Coursera Machine Learning course when working with gradient descent based algorithms is: Debugging gradient descent. Make a plot with number of iterations on the x-...
10k views

### Why my network needs so many epochs to learn?

I'm working on a relation classification task for natural language processing and I have some questions about the learning process. I implemented a convolutional neural network using PyTorch, and I'm ...
6k views

### What are the cases where it is fine to initialize all weights to zero

I've taken a few online courses in machine learning, and in general, the advice has been to choose random weights for a neural network to ensure that your neurons don't all learn the same thing, ...
5k views

### Does MLP always find local minimum

In linear regression we use the following cost function which is a convex function: We Use the following cost function in ...
5k views

### Log loss vs accuracy for deciding between different learning rates?

While model tuning using cross validation and grid search I was plotting the graph of different learning rate against log loss and accuracy separately. Log loss When I used log loss as score in ...
1k views

### The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...
859 views

### Gradient Descent Python Implementation isnt converging

I'm trying to implement gradient descent in Python and following Andrew Ng course in order to follow the math. However, my implementation isn't working as I expected. It would be great if the ...
451 views

I'm a double major in Math and CS interested in Machine Learning. I'm currently taking the popular Coursera course by Prof. Andrew. He's talking and explaining Gradient Descent but I can't avoid ...
658 views

### Meaning of Perceptron optimal weights

Im studying the perceptron algorithm. I know that we can use the weights w as the coefficients of the hyperplane that will separate the vectors that need a classifications In every web page i read ...
468 views

### Gradient descent in a noisy environment

How to know the right direction in a noisy environment? In the typical example of neural network learning, we can see several local minima. The gradient descent is choosing one local minimum and ...
132 views