Questions tagged [overfitting]

Modeling error (especially sampling error) instead of replicable and informative relationships among variables improves model fit statistics, but reduces parsimony, and worsens explanatory and predictive validity.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

How can I prevent overfitting?

hope to find you well ! I am trying to build a model to classiffy customers with propensity to buy, but i cannot get rid of overfitting! My approach is the following: I have created the train dataset ...
0
votes
0answers
15 views

XGBoost: is increasing gamma same as feature selection by average gain?

Since gamma limits splits unless they meet a minimum gain threshold, isn't that the same thing as removing features that have low average gain? Both will results in splits with higher average gains. I ...
0
votes
0answers
19 views

What would cause regression to focus on a few values?

Hi I have a neural network for a regression problem but the results seem to be heavily focused on a few values. I added an image, y axis is predicted values and x axis is real values. What could be ...
0
votes
0answers
24 views

Validation accuracy does not increase for binary classification using GNN

I am trying to perform graph classification with GINConv model of GNN. I have tried everything from varying dropouts to weight decay (for L2 regularization), learning rate (1e-6 to 1e-3), batch ...
0
votes
0answers
8 views

Avoid overfitting in 3 layer neural net for cifar10

I made a neural net with 1 conv and 2 fc layers. I am getting 80-90% accuracy on train set but 60% on train set. I used batchnorm layer too but it didn't remove the overfitting. How can I remove it ...
1
vote
2answers
94 views

Autenocoder and anomaly detection task

I'm trying to create an autoencoder for the anomaly detection task, but I'm noticing that even if it performs very well on the training set, it starts to stop recreating half of the test set. I tried ...
0
votes
0answers
8 views

TF Estimator stop when overfitting?

With TF 1.x is there a way to stop training, when evaluation loss starts increasing? Or is there another way to detect overlearning? I know about the stop_if_no_decrease_hook, but it doesn't consider ...
0
votes
1answer
19 views

Precision, recall and accuracy metrics significantly different between training/validation and actual predictions

I have two sequential models built with Keras that train on data from a CSV file. This is how they are built ...
0
votes
0answers
21 views

Different names for overfitting

If I understand overfitting is simply when our training loss is lower than our test loss. Now let's say we have 2 situations Where your test loss actually starts going up after a certain number of ...
0
votes
0answers
21 views

Neural Net that memorizes what it sees in order?

I'm sorry for this weird question, I know ML is about generalization but I have a specific use case where I'd like to build a neural network or really just a matrix, that memorizes everything it sees ...
1
vote
0answers
16 views

Normal distribution of errors

I'm trying to project lifetime of customers in my company, based on various parameters I've reached a 64% correlation so far, between the valid and prediction data I'm using light GBM regressor I did ...
0
votes
0answers
16 views

Data scaling for convolutional neural networks and other issues

My project involves deep learning on physiological data gathered from a wearable device. I aim to evaluate the potential CNN usefulness in classifying data collected from the wearable. The wearable ...
1
vote
1answer
42 views

Does overfitting depend only on validation loss or both training and validation loss?

There are several scenarios that can occur while training and validating: Both training loss and validation loss are decreasing, with the training loss lower than the validation loss. Both training ...
0
votes
0answers
26 views

Is my LSTM model overfitting or underfitting?

I am currently working on a project to classify comments text into 11 different topics, using a Bidirectional LSTM model. However, the loss curves confuse me as there is a deviation of the training ...
1
vote
0answers
47 views

Data\Feature Leakage - feature too close to target?

In General: The target itself built from very correlated features, because there is no ground truth - only rule based one. I have a problem in the following method: Output: binary. built from ...
0
votes
1answer
52 views

How many features do I select when doing feature selection for regression algorithms? Is R2 and RMSE good measures of success for overfitting?

Context: I'm currently crafting and comparing machine learning models to predict housing data. I have around 32000 data points, 42 features, and I'm predicting housing price. I'm comparing Random ...
1
vote
1answer
35 views

Derive features from test-set?

I have a dataset of choices (between A,B and C) done by certain users, and I want to train a neural network to predict the choices. I divide in train and test sets. An instance is formed by: [UserId, ...
0
votes
0answers
39 views

My Stacked LSTM seems to be doing worse than a shallower one

I started with a two layer LSTM (+ Dense Layers) and which was: ...
1
vote
1answer
31 views

NGBoost and overfit - which model is used?

While training an NGBoost model I got: ...
7
votes
2answers
249 views

Dropping features after final evaluation on test data

Would you please let me know if I am committing a statistical or machine learning mal-practice in this procedure? I want to estimate meteorological variable y1 from ${x_1, ..., x_{10}}$ variables. I ...
1
vote
1answer
49 views

Hyperparameter tunning for Random Forest- choose the best max depth

I'm trying to choose the best parameters for random forest model. For that goal I hae run my model in loop with only one parameter and each time I have changed the number for the parameter max depth. ...
4
votes
2answers
146 views

Trying to figure out which the training set is

Can someone help me on this one? As can be seen in the screenshot, it says the loss is 1/2. Where's that 1/2 coming from? How can I replace the values in the h(s) function? Source PDF
0
votes
0answers
38 views

Overfitting in imbalanced dataset

I am working on a dataset related to an insurance company and the objective is to predict if the insurance buyer will claim their travel insurance or not. Training data: https://raw.githubusercontent....
0
votes
1answer
57 views

Python: How to test a RandomForest regression model for Overfitting?

I'm a beginner in this area so maybe I'm doing something wrong here. I'm using RandomForest for a regression model and wanted to see if my model is overfitting. Here is what I did: EDIT: I use ...
0
votes
1answer
78 views

Number of units for first layer in Keras Sequential Model

I have a huge CSV structured dataset. I'm feeding that dataset to a Keras Sequential Model. My question is, can my Model have number of units greater than the number of input features? At the moment, ...
0
votes
1answer
92 views

why should i do target encoding within cv loop?

i wish to use target encoding, using the category encoders sklearn library. I don't really understand why it is necessary to include this as a step in a sklearn pipeline WITHIN the cross validation ...
0
votes
0answers
14 views

Strange behavior of CNN when forecasting time series

I have a time series containing 5 features. I tried to use LSTM to predict the next 112 periods in the series. However, I got very bad results. So I tried to use CNN. First, it did not work properly ...
0
votes
0answers
73 views

Reduce overfitting in a CNN model

We are Data science students and we are building a CNN model to pneumonia classification (dataset: https://www.kaggle.com/paultimothymooney/chest-xray-pneumonia ). We have applied a data augmentation ...
3
votes
2answers
580 views

Over-sampling: is my model over-fitting?

I would like to ask you some questions on how to consider (good or not) the following results: ...
2
votes
1answer
30 views

machine learning disasters

I am writing a research paper and I am looking for reliable sources that provide information on disasters of machine learning. Especially in the field of autonomous driving. Have there been any ...
8
votes
1answer
278 views

Who invented the concept of over-fitting?

I list the references that I found so far. Shortly, the first appearance of the term was in 1670, first appearance in in close meaning was in 1827, first appearance in a biological paper was in 1923 ...
0
votes
1answer
38 views

Why use regularization?

In a linear model, regularization decreases the slope. Do we just assume that fitting a lin model on training data overfits by almost always creating a slope which is higher than it would be with ...
0
votes
1answer
63 views

Why might my validation loss flatten out while my training loss continues to decrease?

In my effort to learn a bit more about data science I scraped some labeled data from the web and am trying to classify examples into one of three classes. I am running into a problem that, regardless ...
0
votes
0answers
29 views

SVM overfitting with consistent validation results

I have some imbalanced (1400 samples of which 250 are +ve) data for a binary classification problem and I am running an SVM grid search optimising for precision. I am trying 3,4,5,6,7,and 8 stratified ...
0
votes
1answer
39 views

Compare cross validation and test set results

I am having a hard time understanding the results of a cross validation test and a test run on a test set. First I made the following pipeline: ...
2
votes
1answer
208 views

Normal distribution and Random Forest

I have big table in dataframe (600k rows) which has y column (the variable I want to predict) and other 4 other columns that are the X. I have run RF regressor and I got score of 0.87 when I run it ...
4
votes
5answers
2k views

Why does my model produce too good to be true output?

I am trying to run a binary classification problem on people with diabetes and non-diabetes. For labeling my datasets, I followed a simple rule. If a person has T2DM...
1
vote
1answer
40 views

Training Object Detection model on just 10 images

I am trying to train an object detection model using Mask-RCNN with Resnet50 as backbone. I am using the pre-trained models from PyTorch's Torchvision library. I have only 10 images that I can use to ...
2
votes
2answers
54 views

Why is large decision tree likely to overfit

My lecture slide told me that if we don't prune the regression tree, then the tree likely to over-fit. So, I wonder why would that happen? Is that because if the tree grows too large, we would end up ...
2
votes
1answer
41 views

Why Continous Variable Buckets Overfitting model

I have a continuous (high cardinal discrete) variable 'numInteractionPoints' in my dataset during training model - I binned this feature in order to avoid overffing , first top bar chart is from ...
3
votes
1answer
269 views

Determining whether a Machine Learning model is overfitted with regard to the stability of the features

I need to know how would I get to know if I have overfitted my Machine Learning model on the train data. The performance metric I have used is Logistic Loss. Does the stability of the features affect ...
1
vote
0answers
11 views

model tuning by using loss curves

I have been practicing with the following dataset: http://archive.ics.uci.edu/ml/datasets/Concrete+Compressive+Strength for building a prediction model based on a MLP, but I have some doubts if the ...
5
votes
2answers
305 views

Why an increasing validation loss and validation accuracy signifies overfitting?

When I train a neural network, I observe an increasing validation loss, while at the same time, the validation accuracy is also increased. I have read explanations related to the phenomenon, and it ...
1
vote
0answers
37 views

What is the impact of adding a layer in neural networks?

I was playing with hyper-parameters on https://playground.tensorflow.org/ using spiral dataset (classification). So , first I trained a network with 2 hidden layers and the final test and train loss ...
4
votes
1answer
415 views

Can a novelty detection model overfit?

Can a novelty detection model overfit? In novelty detection, the model is trained on normal data instances (not polluted by outliers) where no labels are used in the training process, while validated ...
6
votes
1answer
85 views

XGBoost skews towards minority class

I have a dataset with 85k positive labels and 53k negative labels. For this use-case, I am trying to maximize my efforts to the negative class (accurately identify true negatives, and minimize false ...
0
votes
1answer
46 views

Why does the overfitting decreases if we choose K to be large in K-nearest neighbors?

I am studying machine learning and I am focusing on K-nearest neighbors . I have understood the algorithm, but I have still a doubt, which is on how to choose the K for the number of neighbors. I ...
-1
votes
1answer
36 views

How we can identify the problem of Overfitting and underfitting and maintain bias?

Basically, I'm new to the data science field, and I'm getting a little bit of confusion about overfitting and underfitting. Are overfitting and underfitting is ...
0
votes
1answer
39 views

Regularization hyperparam tuning during training

I have an idea for a regularization-hyperparam selection method, which I haven't encountered before and can't find on Google, but I'm sure someone has already tried it and I'm wondering what are the ...
0
votes
0answers
222 views

SVM is taking too long for hyperparameter tuning

I am running SVM,Logistic Rregression and Random Forest on the credit card dataset. My training dataset has the shape (454491, 30). I performed 5-fold cross validation(which took more than an hour) ...

1
2 3 4 5 6