Skip to main content

Questions tagged [generalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
12 votes
2 answers
4k views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
David Masip's user avatar
  • 6,101
9 votes
2 answers
3k views

High accuracy on test-set, what could go wrong?

You are given a pre-trained binary ML classification model with 99% accuracy on the test-set (assume the customer required 95% and that the test-set is balanced). We would like to deploy our model in ...
CodeHoarder's user avatar
6 votes
1 answer
2k views

Training on accurate data versus noisy data

I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
Mathews24's user avatar
  • 195
5 votes
1 answer
50 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
Sm1's user avatar
  • 541
5 votes
1 answer
2k views

The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...
PickleRick's user avatar
3 votes
1 answer
682 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
ludgo's user avatar
  • 133
2 votes
1 answer
351 views

Which type of models generalize better, generative or discriminative models?

In NLP, which type of models (generative or discriminative) is more sensitive to the amount of data to generalize better? references? This is related to the way those two types capture the data ...
Minions's user avatar
  • 262
2 votes
1 answer
314 views

XGBoost: what to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
FrankBool's user avatar
2 votes
1 answer
431 views

Does convergence equal learning in Deep Q-learning?

In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
Aeryan's user avatar
  • 81
2 votes
1 answer
369 views

Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?

When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4): Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
j200932's user avatar
  • 121
2 votes
1 answer
348 views

Multilabel Classification - Overfitting?

My task is the following: To input drug combinations and output renal failure-related symptoms from the drug combinations. Both the drug combinations and renal-failure related symptoms are represented ...
shepan6's user avatar
  • 1,438
1 vote
4 answers
114 views

How Can I Train a Real-World-Ready Classifier with Limited Real Data and Abundant Open-Source Data?

I am trying to train a text classifier with open-source data to generalize on the real user traffic (henceforth "real data"). However, even though I have many annotated open-source data, I ...
Mr.Robot's user avatar
1 vote
3 answers
99 views

Is my model underfitting?

Model: ...
CJSSSSS's user avatar
  • 33
1 vote
2 answers
328 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
CyberPlayerOne's user avatar
1 vote
1 answer
70 views

What is the form of data used for prediction with generalized stacking ensemble?

I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
rocksNwaves's user avatar
1 vote
1 answer
48 views

How does batch normalization make a model less sensitive to hyperparameter tuning?

Question 22 of 100+ Data Science Interview Questions and Answers for 2022 asks What is the benefit of batch normalization? The first bullet of the answers to this is The model is less sensitive to ...
Galen's user avatar
  • 172
1 vote
2 answers
104 views

Will oversampling help with generalization (small imbalanced dataset)?

I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features. I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
Luis Pinto's user avatar
1 vote
1 answer
630 views

Generalization of RNN/LSTM/GRU... model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
Markus's user avatar
  • 111
1 vote
1 answer
383 views

Multilabel classification: Choosing threshold

I'm creating a multilabel classification approach based on sentence embeddings applied to text taken from a chatbot. We have the following: a training dataset of 2,500 lines, where each line is a ...
LasiusMind's user avatar
1 vote
2 answers
741 views

Graph Neural Network fails at generalizing on unseen graph topologies

I'm using PytorchGeometric to train a graph convolutional network for regression over nodes problem (the graph models physical phenomena in the network of sensors; the network of sensors is actually ...
sesli's user avatar
  • 11
1 vote
0 answers
25 views

Generalization error problem on training set

Training data: $\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$ you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
todapod264's user avatar
1 vote
0 answers
27 views

Reducing High generalization-error on industrial fault data

I have a industrial dataset containing labeled machine data for fault classification(3 classes: 1 ok, 2 for faults). The problem is that i have less (~16) different machines, thus iam currently having ...
deniz's user avatar
  • 51
1 vote
1 answer
192 views

Upper bound on size of sample set for decision trees

Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
Said Savci's user avatar
1 vote
1 answer
1k views

how to shuffle the data for model.fit with custom data generator?

So trainfiles is a list that contains the files' directory and name e.g. ['../train/1.npy' , '../train/2.npy'] and then I create a dataset as shown in the middle of the code then I apply it to model ...
Kevin Kim's user avatar
1 vote
0 answers
21 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
Janne's user avatar
  • 21
1 vote
0 answers
72 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
qwer1304's user avatar
0 votes
2 answers
70 views

Why shouldn't we try to balance the test set?

Most advice I have found online is that we must not balance the test set. The test set should remain to be unseen. However, I failed to see how balancing the test set will cause us to leak knowledge ...
Fraïssé's user avatar
  • 119
0 votes
1 answer
44 views

Why an already trained model is not generalizable to another related dataset?

A model is trained to predict the median temperature of Boston. The resulting model works well according to their validation data. However, this model performs poorly when used to predict the ...
ebrahimi's user avatar
  • 1,307
0 votes
1 answer
56 views

Why does the application of the logarithmic function improve the outcome of Random forests?

I have a Random forest model that tries to predict what kind of a useful activity a machine is doing based on its power readings. There are 5 features in a single reading. There are two main types of ...
Nht_e0's user avatar
  • 21
0 votes
2 answers
842 views

Compare Classification Performance in Datasets drawn from Different Populations

I've read some classics about comparison of ML Algorithms i.e. ...
Björn's user avatar
  • 69
0 votes
2 answers
220 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
HighAsAKiteOnMath's user avatar
0 votes
2 answers
2k views

Why is it bad to use the same test dataset over and over again?

I am following this Google's series: Machine Learning Crash Course. On the chapter about generalisation, they make the following statement: Good performance on the test set is a useful indicator of ...
Bitcoin Cash - ADA enthusiast's user avatar
0 votes
1 answer
36 views

How good are LSTMs in generalizing when learning curves?

I'm interested in the following scenario: I want to learn a mapping that maps a function to another function, i.e. I want to approximate a functional operator. If one is unfimiliar with operators one ...
ZenDen's user avatar
  • 1
0 votes
1 answer
37 views

Is it possible to train a Support Vector Machine to a specific accuracy?

From my understanding, support vector machines run on the premise of minimizing some error function, usually with the goal of maximizing accuracy overall. However, there are a lot of contexts, ...
PSB's user avatar
  • 27
0 votes
1 answer
12 views

Doing correctly paired-trial validation

In paired-trial validation, a statistical (ML) models are trained on $n$ datasets separately and then applied to other datasets, as a way of estimating the generalization of the models obtained. ...
Roger V.'s user avatar
  • 103
0 votes
1 answer
23 views

How to add lagging Features for Forecasting with a random lag range without adding a new column per lag?

The common way of adding lagging features in time series forecasting problems is adding lag columns with pandas.shift(). While it is a fine method but what about when wanting to use a random integer (...
Emad Ezzeldin's user avatar
0 votes
0 answers
29 views

What if many different models reach the same maximum metrics

I am talking about trying different algorithms, different parameters, different stacking configurations that all improve upon previous baselines, and yet a lot of them have exactly the same values on ...
liakoyras's user avatar
  • 636
0 votes
1 answer
46 views

About neural network ability to generalize

I had this question during an interview that I wasn't able to answer, even after researching on the internet. Which of the following can affect an artificial neural network’s ability to generalize??? ...
user131341's user avatar
0 votes
0 answers
28 views

Can there be scenarios where an overfitted model in machine learning cannot be generalized?

Is it always possible to generalize an overfitted model? I know there are ways to handle overfitting, but can there be scenarios where overfitting cannot be handled in machine learning?
user16584277's user avatar
0 votes
1 answer
236 views

Why when my local cv of loss decreases, my leaderboard's loss increases?

I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514. I further did feature engineering and added some features to the ...
Haripriya R's user avatar
0 votes
1 answer
70 views

Are there some research papers about text-to-set generation?

I have googled but find no results. Text-to-(word)set generation or sequence-to-(token)set generation. For example, input a text and then output the tags for this text: ...
CoderOnly's user avatar
  • 711
0 votes
1 answer
42 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
Shree Singhi's user avatar
0 votes
1 answer
220 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
Joe B's user avatar
  • 322