# Questions tagged [convergence]

The tag has no usage guidance.

38 questions
Filter by
Sorted by
Tagged with
40 views

### How big is the threshold that is usually used in determining the convergence of loss values in deep learning?

In deep learning, one way to determine whether the training has converged is to observe the movement of the loss values over iterations or epochs. One can choose any $\epsilon$ threshold and any ...
1 vote
101 views

### What is the definition of convergence in the context of deep neural networks?

Suppose I have a feed forward neural network which approximates a value, say $Y_0$. The analytical value of $Y_0$ is given. The plot of the network approximation of $Y_0$ each step is given as follows....
298 views

### Pytorch Neural Network that tries to approximate $z_i = x_i^2 + y_i^2$ not converging to solution

Background I am teaching myself Pytorch, as a Mechanical engineering technology (MET) faculty. My end goal is to replace many data-driven heat transfer and Fluid dynamics models with Neural network ...
1 vote
145 views

### How to calculate the KL divergence for two multivariate pandas dataframes

I am training a Gaussian-Process model iteratively. In each iteration, a new sample is added to the training dataset (Pandas DataFrame), and the model is re-trained and evaluated. Each row of the ...
49 views

### Is batch size of 1 a valid choice for a very deep neural network with high memory requirement?

I am training a very deep neural network (Panoptic-DeepLab) with a ResNet34 backbone on Google Colab on CityScapes dataset for Panoptic Segmentation, and noticed that, with a big crop size, the batch ...
2k views

### What exactly is convergence rate referring to in machine learning?

My understanding of the term "Convergence Rate" is as follows: Rate at which maximum/Minimum of a function is reached, so in logistic regression rate at which gradient decent reaches global ...
39 views

### What is going on with this kind of validation loss graph?

I am using stock prices and a whole bunch of indicators values to try to get a tensorflow model to predict to buy,sell, or hold. I think im going about this right but when i train the model, first i ...
172 views

### Rate of convergence - comparison of supervised ML methods

I am working on a project with sparse labelled datasets, and am looking for references regarding the rate of convergence of different supervised ML techniques with respect to dataset size. I know that ...
294 views

### Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
1 vote
2k views

### Do smaller neural nets always converge faster than larger ones?

In your experience, do smaller CNN models (fewer params) converge faster than larger models? I would think yes, naturally, because there are fewer parameters to optimize. However, I am training a a ...
31 views

### Force Matching in Coarse Grained Molecular Dynamics with Jax - Forces do not match when neglecting energy loss

I am currently exploring force matching approaches for molecular dynamic simulations. As I am still in an exploration state, I'd tried investigated Force Matching Neural Network Colab Notebook ...
161 views

### ElasticNet Convergence odd behavior

I am optimizing a model using ElasticNet, but am getting some odd behavior. When I set the tolerance hyperparameter with a small value, I get ...
1 vote
68 views

### Uniform convergence garantee on sample complexity

I can't understand why the Uniform Convergence guarantees an upper bound and not a lower bound on sample complexity as stated on  Corollary 4.4. If a class $H$ has the uniform convergence property ...
17k views

### Logistic regression does cannot converge without poor model performance

I have a multi-class classification logistic regression model. Using a very basic sklearn pipeline I am taking in cleansed text descriptions of an object and classifying said object into a category. <...
396 views

137 views

### need explanation on how an equation is being converted to cvxopt logic in solver.lq

This is the equation that is given in the example: and the code to replicate it in python is ...
13k views

### Is the percepetron algorithm's convergence dependent on the linearity of the data?

Does the fact that I have linearly separable data or not impact the convergence of the perceptron algorithm? Is it always gonna converge if the data is linearly separable and not if it is not ? Is ...