All Questions
Tagged with regularization overfitting
23 questions
0
votes
0
answers
24
views
Question about the limitations of regularization
I am training a neural network which is overfitting. Even when I increase the number of parameters, the test lost plateaus while the training loss keeps decreasing. Can regularization (like an L1 or ...
0
votes
0
answers
49
views
Using Embedding For Regularization
Is using embeddings for regularization a valid practice?
My reasoning for that is that encoding training/tests datasets into smaller vectors would allow a smaller network with fewer parameters and ...
1
vote
0
answers
14
views
State-of-the-art techniques for regularizing Neural Networks?
For regularizing neural networks, I'm familiar with drop-out and l2/l1 regularization, which were the biggest players in the late 2010's.
Have any significant/strong competitors risen up since then?
- 111
3
votes
1
answer
52
views
Request: Confirmation on my understanding of overfitting and regularization concepts
Overfitted models tend to have largely different (some very high, some comparatively low) coefficients/weights for different feature values. So, this means the model (when drawn as graph) will have ...
- 161
0
votes
1
answer
42
views
1
vote
0
answers
33
views
Understanding model's learning curves
I'm trying to train a Lane Detection CNN called PINet on a proprietary dataset.
Below are some of the important configuration values:
Batch size: 6
Optimizer: Adam
Learning rate: High of 1e-4 and Low ...
- 11
0
votes
1
answer
74
views
Why use regularization?
In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with ...
- 3
1
vote
1
answer
354
views
Do non-parametric models always overfit without regularization?
Let's scope this to just classification.
It's clear that if you fully grow out a decision tree with no regularization (e.g. max depth, pruning), it will overfit the training data and get full accuracy ...
- 111
1
vote
1
answer
62
views
regularization error vs over fitting
I have a collected data from 50 unique blocks, and then merged data from 49 blocks into one data set, and saved the data from 1 block for testing purpose.
I then split the merged data set from 49 ...
2
votes
1
answer
5k
views
How to reduce overfitting in a pre-trained network
I have a custom dataset with 10 classes and I am using a pre-trained resnet18 model from torch-vision. I can clearly see it's over-fitting because: the model is trained for 75 epochs with a batch size ...
- 143
4
votes
2
answers
3k
views
How to handle Overfitting
I am working on machine learning classification problem with two classes (0/1). I would like to build a prediction model. The problem is that I have a small dataSet of ...
- 103
-1
votes
1
answer
86
views
How does regularization help?
What is the effect of regularization on the value of parameters/weights?
How does adding a regularization term in the cost function(J) and gradients help?
Doesn't adding something increase the cost ...
1
vote
2
answers
69
views
Over fitting and association with regularization
Heard and read lot about regularization helps in reducing over fitting.
But I'm not sure how exactly regularization works in reducing over fitting issue and whats the maths behind it?
Appreciate if ...
user91244
2
votes
2
answers
198
views
How to build an overfitted network in order to increase performances
I am learning how to implement CNN, and searching on the internet I have found that a trick to design a good network is to first build it in such a way that it overfits, and then use regularization to ...
- 921
5
votes
2
answers
4k
views
If my model is overfitting the training dataset, does adding noise to training dataset help regularizing the machine learning model
I would like to know if this is a best practice or not. Can we add noise to the training data to help the model "fit less the training data"; as a result, hoping to generalize better on new unseen ...
- 269
1
vote
0
answers
265
views
High Variance on CNN
I'm using a shallow CNN for my current project [this one]. I have a training dataset consisting of 1000 samples and a test dataset of 400 samples. I'm using the test dataset to choose the best ...
- 81
1
vote
1
answer
372
views
Improving Accuracy of the Deep Learning Model
In my current project, I have only 647 rows (500 for training and 147 for testing) and I have applied the Keras Sequential model using the following code:
...
- 123
1
vote
3
answers
205
views
What is the point of getting rid of overfitting?
I'm having trouble understanding why I would use dropout, regularization, data augmentation, etc to get rid of overfitting in the first place. I get that if your model is too large or data is too ...
- 191
32
votes
2
answers
26k
views
When should one use L1, L2 regularization instead of dropout layer, given that both serve same purpose of reducing overfitting?
In Keras, there are 2 methods to reduce over-fitting. L1,L2 regularization or dropout layer.
What are some situations to use L1,L2 regularization instead of dropout layer? What are some situations ...
- 1,445
1
vote
1
answer
367
views
Loss for CNN decreases and settles but training accuracy does not improve
I am training a CNN with 2 conv layers 2 Relu and max pooling and 2 FC layers the last of which has only 2 units since it's a binary classification problem. The images are spatio-temporal continuous, ...
- 90
8
votes
1
answer
7k
views
Dropout vs weight decay
Dropout and weight decay are both regularization techniques. From my experience, dropout has been more widely used in the last few years. Are there scenarios where weight decay shines more than ...
- 6,106
1
vote
1
answer
242
views
Should I set higher dropout prob if there are plenty of data?
I have some excessive amount of data for the size of NN I am able to teach in a reasonable time.
If I feed all the data into the network it stops learning at some point and a resulting model shows ...
- 281
1
vote
2
answers
99
views
How can I fix this "convex" problem ? Is it just a matter of overfitting?
I get some metrics on validation data while training a model , and in my case the they are :
(0.25, 0.31, 0.46, 0.57, 0.65, 0.75, 0.77, 0.78, 0.84, 0.84, 0.85,
0.84, 0.84, 0.84, 0.82, 0.8, 0.8, 0....
- 399