Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
0answers
9 views

Will repeatedly fine-tuning on new data cause overfitting?

I have a binary classification model which I have trained on a training set. On the validation set its accuracy is ~85%. I set up early stopping which ended training when validation loss increased. ...
1
vote
2answers
50 views
+50

Is it possible to use a Neural Network to interpolate data?

I am completely new to Artificial intelligence and Neural Networks. I am currently working on a plasma physics simulation project which requires a very high resolution data set. We currently have the ...
2
votes
0answers
19 views

When to stop the final model training?

Let's say I'm participating in a Kaggle image recognition competition. Firstly, I create a train/validation split and find the good hyperparameters for my model. Here the stopping criterion is when ...
0
votes
0answers
13 views

Systematically finding a CNN architecture?

I am trying to train a classifier from 25k images and 7k classes. Seems like my model overfits just after 3 epochs. I have tried to reduce the model complexity and increase the weight decay but still, ...
1
vote
0answers
22 views

Random Forest overfitting with `n_estimators=1`, `max_depth=1 and `max_features =1`

I am trying to stop my RF from overfitting. I am using time series data with 1 day time lag, to predict the current price. I am using this function to shift my independent features back 1 day: ...
4
votes
2answers
210 views

Overfitting in active learning

How can I make sure that the initial model trained on a small dataset will not suffer from overfitting before applying the active learning sampling techniques? because I will use this model to select ...
0
votes
0answers
34 views

Is my CNN model overfitting or underfitting?

I would like to be sure of whether the model is overfitting or undercutting. Being new to this, is there any specific point to identify when to stop the training process. Any help in this regard would ...
2
votes
1answer
49 views

How to analyze neural network quality in case of overfitting?

I have a Keras neural network that has images both as input and reference data. My network demonstrates overfitting (for example, train accuracy is about 80% but test accuracy is only up to 70%) due ...
1
vote
0answers
29 views

CNN + LSTM model for images performs poorly on validation data set

My training and loss curves look like below and yes, similar graphs have received comments like "Classic overfitting" and I get it. My model looks like below, ...
1
vote
0answers
44 views

XGBoost regressor hyperparameter tuning with hyperopt leads to overfit

Using hyperopt to hyperparameter tuning on XGBoost regressor, I am receiving overfiting on the train set. There is any suggestion how to solve it ? I have used cross validation with ...
0
votes
0answers
15 views

How to improve SGD classifier performance?

I am having some issues with SGD to run with a sample of approx 5000 obs for a classification problem with imbalance. I am doing ...
0
votes
1answer
179 views

How to perform Multi-Label Image Classification with EfficientNet

Problem My goal is to perform multi-label image classification with EfficientNet. It should take a picture as input and e.g. tell the user that it sees a person AND a dog on the picture, meaning the ...
0
votes
0answers
36 views

Unet Overfitting for binary segmentation of fake images

I am working on a project where I am trying to detect and localize forgeries in images. I am using the CASIA v2 dataset and using Unet model for the task. I have the binary masks of all the images in ...
1
vote
0answers
22 views

Understanding model's learning curves

I'm trying to train a Lane Detection CNN called PINet on a proprietary dataset. Below are some of the important configuration values: Batch size: 6 Optimizer: Adam Learning rate: High of 1e-4 and Low ...
1
vote
2answers
49 views

ideal algorithms to demonstrate overfitting or underfitting

When one tries to look up concepts such as overfitting and underfitting, the most common thing that pops up is polynomial regression. Why is polynomial regression often used to demonstrate these ...
0
votes
1answer
21 views

Accuracy on Validation and Test set, Overfit?

Just a quick question, I am building a ML model right now however I am receiving very similar (72.2 and 72.4 for example)% for both Accuracy and F1-Score on my Validation Dataset and my unseen Test ...
0
votes
1answer
15 views

Avoiding overfitting in unsupervised ML

I am using a unsupervised pattern matching approach to create a trade strategy. I use the output of the pattern matched results to decide whether to enter a trade or not. For deciding the best pattern ...
0
votes
0answers
34 views

How many epochs should all the data be trained on after training with validation finds when validation and training diverge?

One uses and train/test split to use their training data to get an idea of how many epochs to train with. If the validation accuracy starts going down while the training accuracy is still going up, ...
1
vote
0answers
92 views

BERT MLM overfitting [closed]

We are training the BERT model on masked language modeling task for the Russian Language. Our dataset consists of 60 mln texts with (128 tokens for each text) from online social networks, ...
0
votes
1answer
21 views

Performance metrics changing significantly based on batch size

I am working on a binary classification problem where there is significant class imbalance (minority class makes up nearly 10%). The dataset has ~15,000 observations and I have split this in to a ...
1
vote
2answers
32 views

dataset split for image classification

I am trying to do image classification for 14 categories (around 1000 images for each cat). And i initially created two folders for training and validation. In this case, do I still need to set a ...
0
votes
0answers
26 views

How can I prevent overfitting?

hope to find you well ! I am trying to build a model to classiffy customers with propensity to buy, but i cannot get rid of overfitting! My approach is the following: I have created the train dataset ...
0
votes
0answers
15 views

XGBoost: is increasing gamma same as feature selection by average gain?

Since gamma limits splits unless they meet a minimum gain threshold, isn't that the same thing as removing features that have low average gain? Both will results in splits with higher average gains. I ...
0
votes
0answers
22 views

What would cause regression to focus on a few values?

Hi I have a neural network for a regression problem but the results seem to be heavily focused on a few values. I added an image, y axis is predicted values and x axis is real values. What could be ...
0
votes
0answers
30 views

Validation accuracy does not increase for binary classification using GNN

I am trying to perform graph classification with GINConv model of GNN. I have tried everything from varying dropouts to weight decay (for L2 regularization), learning rate (1e-6 to 1e-3), batch ...
0
votes
0answers
9 views

Avoid overfitting in 3 layer neural net for cifar10

I made a neural net with 1 conv and 2 fc layers. I am getting 80-90% accuracy on train set but 60% on train set. I used batchnorm layer too but it didn't remove the overfitting. How can I remove it ...
1
vote
2answers
111 views

Autenocoder and anomaly detection task

I'm trying to create an autoencoder for the anomaly detection task, but I'm noticing that even if it performs very well on the training set, it starts to stop recreating half of the test set. I tried ...
0
votes
0answers
15 views

TF Estimator stop when overfitting?

With TF 1.x is there a way to stop training, when evaluation loss starts increasing? Or is there another way to detect overlearning? I know about the stop_if_no_decrease_hook, but it doesn't consider ...
0
votes
1answer
39 views

Precision, recall and accuracy metrics significantly different between training/validation and actual predictions

I have two sequential models built with Keras that train on data from a CSV file. This is how they are built ...
0
votes
0answers
21 views

Different names for overfitting

If I understand overfitting is simply when our training loss is lower than our test loss. Now let's say we have 2 situations Where your test loss actually starts going up after a certain number of ...
0
votes
0answers
23 views

Neural Net that memorizes what it sees in order?

I'm sorry for this weird question, I know ML is about generalization but I have a specific use case where I'd like to build a neural network or really just a matrix, that memorizes everything it sees ...
1
vote
0answers
19 views

Normal distribution of errors

I'm trying to project lifetime of customers in my company, based on various parameters I've reached a 64% correlation so far, between the valid and prediction data I'm using light GBM regressor I did ...
0
votes
0answers
19 views

Data scaling for convolutional neural networks and other issues

My project involves deep learning on physiological data gathered from a wearable device. I aim to evaluate the potential CNN usefulness in classifying data collected from the wearable. The wearable ...
1
vote
1answer
54 views

Does overfitting depend only on validation loss or both training and validation loss?

There are several scenarios that can occur while training and validating: Both training loss and validation loss are decreasing, with the training loss lower than the validation loss. Both training ...
0
votes
0answers
51 views

Is my LSTM model overfitting or underfitting?

I am currently working on a project to classify comments text into 11 different topics, using a Bidirectional LSTM model. However, the loss curves confuse me as there is a deviation of the training ...
1
vote
0answers
48 views

Data\Feature Leakage - feature too close to target?

In General: The target itself built from very correlated features, because there is no ground truth - only rule based one. I have a problem in the following method: Output: binary. built from ...
0
votes
1answer
128 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
1
vote
1answer
37 views

Derive features from test-set?

I have a dataset of choices (between A,B and C) done by certain users, and I want to train a neural network to predict the choices. I divide in train and test sets. An instance is formed by: [UserId, ...
0
votes
0answers
51 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
1
vote
1answer
41 views

NGBoost and overfit - which model is used?

While training an NGBoost model I got: ...
7
votes
2answers
254 views

Dropping features after final evaluation on test data

Would you please let me know if I am committing a statistical or machine learning mal-practice in this procedure? I want to estimate meteorological variable y1 from ${x_1, ..., x_{10}}$ variables. I ...
1
vote
1answer
65 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
4
votes
2answers
147 views

Trying to figure out which the training set is

Can someone help me on this one? As can be seen in the screenshot, it says the loss is 1/2. Where's that 1/2 coming from? How can I replace the values in the h(s) function? Source PDF
0
votes
0answers
52 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
0
votes
1answer
63 views

Python: How to test a RandomForest regression model for Overfitting?

I'm a beginner in this area so maybe I'm doing something wrong here. I'm using RandomForest for a regression model and wanted to see if my model is overfitting. Here is what I did: EDIT: I use ...
0
votes
1answer
105 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0
votes
1answer
137 views

why should i do target encoding within cv loop?

i wish to use target encoding, using the category encoders sklearn library. I don't really understand why it is necessary to include this as a step in a sklearn pipeline WITHIN the cross validation ...
0
votes
0answers
16 views

Strange behavior of CNN when forecasting time series

I have a time series containing 5 features. I tried to use LSTM to predict the next 112 periods in the series. However, I got very bad results. So I tried to use CNN. First, it did not work properly ...
0
votes
0answers
76 views

Reduce overfitting in a CNN model

We are Data science students and we are building a CNN model to pneumonia classification (dataset: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia ). We have applied a data augmentation ...
3
votes
2answers
584 views

Over-sampling: is my model over-fitting?

I would like to ask you some questions on how to consider (good or not) the following results: ...

1
2 3 4 5 6