Questions tagged [convergence]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
19
votes
2answers
5k views

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
14
votes
4answers
15k views

Number of epochs in Gensim Word2Vec implementation

There's an iter parameter in the gensim Word2Vec implementation class gensim.models.word2vec.Word2Vec(sentences=None, size=...
3
votes
4answers
170 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
3
votes
1answer
119 views

need explanation on how an equation is being converted to cvxopt logic in solver.lq

This is the equation that is given in the example: and the code to replicate it in python is ...
2
votes
1answer
3k views

Why this model does not converge in keras?

This case has an underlying story but I have essentially boiled it down to the simplest possible re-producible example I could. Essentially let us think that I have up to 1000 nodes and each node ...
2
votes
1answer
37 views

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
2
votes
1answer
715 views

Very slow convergence with CNN [closed]

I am new to deep learning. I am working on training an SSD model on a set of small objects. I am using Adam gradient descent for optimization and a large input (800x800), but I seem to only get an ...
2
votes
0answers
21 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
2
votes
0answers
55 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
2
votes
0answers
569 views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
2
votes
1answer
519 views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
1
vote
1answer
25 views

Why GA convergence curves continue as two parallel lines?

I'm working on a optimization problem and using GA algorithm (in MATLAB, ga function). As you know MATLAB plots GA result with two curves, one for the best values and other to show the mean values ...
1
vote
1answer
5k views

Is the percepetron algorithm's convergence dependent on the linearity of the data?

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...
1
vote
1answer
1k views

Recurrent Neural Network (LSTM) not converging during optimization

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
1
vote
1answer
1k views

Convergence of vanilla or natural policy gradients (e.g. REINFORCE)

I am reading in a lot of places that policy gradients, especially vanilla and natural, are at least guaranteed to converge to a local optimum (see, e.g., pg. 2 of Policy Gradient Methods for Robotics ...
1
vote
0answers
41 views

Help with MLP convergence

I posted this question on AI SE and got advised to ask here for guidance. I've been stuck for a couple of days trying to figure it out how the standard MLP works and why my code doesn't converge at ...
1
vote
0answers
48 views

Huge cost not converging well with TensorFlow logistic regression

I try to use Logistic Regression for a dataset which contains 15 numeric features and 4238 rows of examples. The calculated cost started at 415.91, and converged when the cost was reduced to 220.119 ...
1
vote
0answers
232 views

sequence convergence - python

Let $\{p_i : i \in \mathbb{Z}\}$ be an i.i.d sequence with $p_i \in (0, 1)$. Fix this random environment, then consider the random walk $$P[ X_{n+1} = i + 1 | X_n = i] = 1 − P[ X_{n+1} = i − 1 | X_n =...
1
vote
0answers
548 views

Validation loss keeps fluctuating about training loss

I am training a Keras model for multi-target regression by using a custom loss function with the goal of getting predictions accurate to below 0.01 with respect to ...
0
votes
2answers
27 views

How to assess that a cross entropy based model has converged

I have a question regarding cross entropy convergence using Stochastic Gradient Descent. I am a little bit confused about how the convergence should be assessed. Should we treat the model as converged ...
0
votes
1answer
15 views

How do i iterate functions until convergence in R?

I am looking to iterate until convergence but I am not sure how i should code it. A simple similar example is something like $X_0=5 , Y_0=3$ for $n=1,2..$ $P_n=X_{n-1}-Y_{n-1}$ $Y_n=3P_n$ $X_n=P_n+...
0
votes
2answers
412 views

Python lifelines - ConvergenceWarning: Newton-Raphson failed to converge sufficiently in Cox prop hazard

When calling CoxPHFitter() on my full dataset I'm getting the following error: ...
0
votes
1answer
57 views

Can one use non converged results from Logistic Regression?

I'm running Logistic Regression on a dataset for a classification problem. I used the model on the dataset when it was normalized and I had no problem with it converging. Now, I wanted to see the ...
0
votes
0answers
13 views

Convergence check of constrained biconvex optimization problem

I participate in the development of a matrix factorization algorithm and I have some convergence issues. Here is the kind of minimization problem I am facing : $L(D,A) = ||X-DA||^2 + \sum_{(i,j) \in ...
0
votes
0answers
89 views

Cross entropy loss for sigmoid function

Suppose instead of squared error loss I take cross entropy loss: $$H(y) = - \sum y' \log(y) $$ ( where y' is the actual distribution) . I read somewhere that this loss function converges faster ...