Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [convergence]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
1 vote
0 answers
7 views

Checking MixedModel convergence in Julia

I'm trying to teach myself how to use Julia as an alternative to R, and was running into some trouble with checking convergence of a MixedModel object. In R, if I use the following code ...
Vincent's user avatar
  • 11
0 votes
0 answers
8 views

kernal failure on data science ICA algorithm for independent component analysis

ICA algorithm Problem Statement The goal is to have the ICA algorithm for independent component analysis custom python code perform convergence on many independent audio wave files, e.g., 10, 15, 20, ...
Data Science Analytics Manager's user avatar
0 votes
0 answers
27 views

Training rate of "convergence" vs viability of network architecure

I am still fairly new to modern neural networks (dabbled in the 90's when convolution was called neocognition which I didn't explore regrettably). I read the suggestions after typing in the above ...
user2624395's user avatar
0 votes
2 answers
69 views

Is improving a Neural Network really just "trial and error"?

After asking on StackOverflow, I was redirected here, so I'm reposting this question. I am a PhD student in Computational Physics and I've started to study a bit of Neural Networks, and decided to try ...
Mauro Giliberti's user avatar
0 votes
1 answer
50 views

How big is the threshold that is usually used in determining the convergence of loss values in deep learning?

In deep learning, one way to determine whether the training has converged is to observe the movement of the loss values over iterations or epochs. One can choose any $\epsilon$ threshold and any ...
poglhar's user avatar
  • 25
1 vote
1 answer
264 views

What is the definition of convergence in the context of deep neural networks?

Suppose I have a feed forward neural network which approximates a value, say $Y_0$. The analytical value of $Y_0$ is given. The plot of the network approximation of $Y_0$ each step is given as follows....
poglhar's user avatar
  • 25
2 votes
1 answer
366 views

Pytorch Neural Network that tries to approximate $z_i = x_i^2 + y_i^2$ not converging to solution

Background I am teaching myself Pytorch, as a Mechanical engineering technology (MET) faculty. My end goal is to replace many data-driven heat transfer and Fluid dynamics models with Neural network ...
dearN's user avatar
  • 123
1 vote
0 answers
192 views

How to calculate the KL divergence for two multivariate pandas dataframes

I am training a Gaussian-Process model iteratively. In each iteration, a new sample is added to the training dataset (Pandas DataFrame), and the model is re-trained and evaluated. Each row of the ...
guest001's user avatar
2 votes
1 answer
70 views

Is batch size of 1 a valid choice for a very deep neural network with high memory requirement?

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch ...
A_C's user avatar
  • 121
0 votes
1 answer
2k views

What exactly is convergence rate referring to in machine learning?

My understanding of the term "Convergence Rate" is as follows: Rate at which maximum/Minimum of a function is reached, so in logistic regression rate at which gradient decent reaches global ...
haneulkim's user avatar
  • 469
0 votes
0 answers
40 views

What is going on with this kind of validation loss graph?

I am using stock prices and a whole bunch of indicators values to try to get a tensorflow model to predict to buy,sell, or hold. I think im going about this right but when i train the model, first i ...
JRowan's user avatar
  • 101
2 votes
1 answer
202 views

Rate of convergence - comparison of supervised ML methods

I am working on a project with sparse labelled datasets, and am looking for references regarding the rate of convergence of different supervised ML techniques with respect to dataset size. I know that ...
Avatrin's user avatar
  • 131
2 votes
1 answer
424 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
Aeryan's user avatar
  • 81
1 vote
2 answers
2k views

Do smaller neural nets always converge faster than larger ones?

In your experience, do smaller CNN models (fewer params) converge faster than larger models? I would think yes, naturally, because there are fewer parameters to optimize. However, I am training a a ...
Sid's user avatar
  • 677
3 votes
0 answers
37 views

Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook ...
not_converging's user avatar
3 votes
1 answer
188 views

ElasticNet Convergence odd behavior

I am optimizing a model using ElasticNet, but am getting some odd behavior. When I set the tolerance hyperparameter with a small value, I get ...
mr_python's user avatar
1 vote
0 answers
72 views

Uniform convergence garantee on sample complexity

I can't understand why the Uniform Convergence guarantees an upper bound and not a lower bound on sample complexity as stated on [1] Corollary 4.4. If a class $H$ has the uniform convergence property ...
Enrico R.'s user avatar
5 votes
2 answers
19k views

Logistic regression does cannot converge without poor model performance

I have a multi-class classification logistic regression model. Using a very basic sklearn pipeline I am taking in cleansed text descriptions of an object and classifying said object into a category. <...
jasper's user avatar
  • 73
0 votes
1 answer
429 views

How do i iterate functions until convergence in R?

I am looking to iterate until convergence but I am not sure how i should code it. A simple similar example is something like $X_0=5 , Y_0=3$ for $n=1,2..$ $P_n=X_{n-1}-Y_{n-1}$ $Y_n=3P_n$ $X_n=P_n+...
John_cena123123's user avatar
0 votes
2 answers
267 views

How to assess that a cross entropy based model has converged

I have a question regarding cross entropy convergence using Stochastic Gradient Descent. I am a little bit confused about how the convergence should be assessed. Should we treat the model as converged ...
Jim's user avatar
  • 125
1 vote
1 answer
153 views

Why GA convergence curves continue as two parallel lines?

I'm working on a optimization problem and using GA algorithm (in MATLAB, ga function). As you know MATLAB plots GA result with two curves, one for the best values and other to show the mean values ...
Davood's user avatar
  • 221
2 votes
0 answers
26 views

Is there a universal convergence rate when stacking models/experts?

It's fairly common to see people stacking different models when chasing marginal gains in contexts such as Kaggle competitions or the Netflix challenge. I would like to know about the mathematics ...
Learning is a mess's user avatar
2 votes
1 answer
114 views

How can we conclude that an optimization algorithm is better than another one

When we test a new optimization algorithm, what the process that we need to do?For example, do we need to run the algorithm several times, and pick a best performance,i.e., in terms of accuracy, f1 ...
user82620's user avatar
1 vote
0 answers
202 views

Help with MLP convergence

I posted this question on AI SE and got advised to ask here for guidance. I've been stuck for a couple of days trying to figure it out how the standard MLP works and why my code doesn't converge at ...
ivansnpmaster's user avatar
0 votes
2 answers
1k views

Python lifelines - ConvergenceWarning: Newton-Raphson failed to converge sufficiently in Cox prop hazard

When calling CoxPHFitter() on my full dataset I'm getting the following error: ...
Serendipity's user avatar
1 vote
0 answers
56 views

Huge cost not converging well with TensorFlow logistic regression

I try to use Logistic Regression for a dataset which contains 15 numeric features and 4238 rows of examples. The calculated cost started at 415.91, and converged when the cost was reduced to 220.119 ...
user avatar
2 votes
0 answers
307 views

How does permutation of training data improve convergence time when training a perceptron or neural network model? [duplicate]

I'm currently studying some basic concepts regarding Deep Learning and Neural Networks with this material. When discussing the training algorithm for a perceptron, the author states that looping ...
Nilton Junior's user avatar
0 votes
1 answer
310 views

Can one use non converged results from Logistic Regression?

I'm running Logistic Regression on a dataset for a classification problem. I used the model on the dataset when it was normalized and I had no problem with it converging. Now, I wanted to see the ...
dungeon's user avatar
  • 175
1 vote
0 answers
693 views

sequence convergence - python

Let $\{p_i : i \in \mathbb{Z}\}$ be an i.i.d sequence with $p_i \in (0, 1)$. Fix this random environment, then consider the random walk $$P[ X_{n+1} = i + 1 | X_n = i] = 1 − P[ X_{n+1} = i − 1 | X_n =...
Med Qadi's user avatar
2 votes
0 answers
1k views

LSTM not converging

I am sorry if this questions is basic but I am quite new to NN in general. I am trying to build an LSTM to predict certain properties of a light curve (the output is 0 or 1). I build it in pytorch. ...
Bill's user avatar
  • 21
4 votes
1 answer
1k views

DQN fails to find optimal policy

Based on DeepMind publication, I've recreated the environment and I am trying to make the DQN find and converge to an optimal policy. The task of an agent is to learn how to sustainably collect apples ...
macwiatrak's user avatar
3 votes
4 answers
556 views

Weights not converging while cost function has converged in neural networks

My cost/loss function drops drastically and approaches 0, which looks a sign of convergence. But the weights are still changing in a visible way, a lot faster than the cost function. Should I ensure ...
feynman's user avatar
  • 237
1 vote
1 answer
2k views

Recurrent Neural Network (LSTM) not converging during optimization

I am trying to train a RNN with text from wikipedia but I having having trouble getting the RNN to converge. I have tried increasing the batch size but it doesn't seem to be helping. All data is one ...
treutm's user avatar
  • 37
2 votes
1 answer
2k views

Very slow convergence with CNN [closed]

I am new to deep learning. I am working on training an SSD model on a set of small objects. I am using Adam gradient descent for optimization and a large input (800x800), but I seem to only get an ...
deeprai's user avatar
  • 23
2 votes
1 answer
4k views

Why this model does not converge in keras?

This case has an underlying story but I have essentially boiled it down to the simplest possible re-producible example I could. Essentially let us think that I have up to 1000 nodes and each node ...
Aliostad's user avatar
  • 135
1 vote
1 answer
863 views

Validation loss keeps fluctuating about training loss

I am training a Keras model for multi-target regression by using a custom loss function with the goal of getting predictions accurate to below 0.01 with respect to ...
a_guest's user avatar
  • 111
3 votes
1 answer
3k views

Convergence of vanilla or natural policy gradients (e.g. REINFORCE)

I am reading in a lot of places that policy gradients, especially vanilla and natural, are at least guaranteed to converge to a local optimum (see, e.g., pg. 2 of Policy Gradient Methods for Robotics ...
user650261's user avatar
3 votes
2 answers
655 views

Normalizing the final weights vector in the upper bound on the Perceptron's convergence

The convergence of the "simple" perceptron says that: $$k\leqslant \left ( \frac{R\left \| \bar{\theta} \right \|}{\gamma } \right )^{2}$$ where $k$ is the number of iterations (in which the weights ...
Qwerto's user avatar
  • 705
3 votes
1 answer
140 views

need explanation on how an equation is being converted to cvxopt logic in solver.lq

This is the equation that is given in the example: and the code to replicate it in python is ...
Sourav Roy's user avatar
3 votes
1 answer
13k views

Is the percepetron algorithm's convergence dependent on the linearity of the data?

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...
astudentofmaths's user avatar
28 votes
2 answers
13k views

local minima vs saddle points in deep learning

I heard Andrew Ng (in a video I unfortunately can't find anymore) talk about how the understanding of local minima in deep learning problems has changed in the sense that they are now regarded as less ...
oW_'s user avatar
  • 6,377
17 votes
5 answers
26k views

Number of epochs in Gensim Word2Vec implementation

There's an iter parameter in the gensim Word2Vec implementation ...
alvas's user avatar
  • 2,410