Questions tagged [generalization]
The generalization tag has no usage guidance.
40
questions
1
vote
4
answers
106
views
How Can I Train a Real-World-Ready Classifier with Limited Real Data and Abundant Open-Source Data?
I am trying to train a text classifier with open-source data to generalize on the real user traffic (henceforth "real data"). However, even though I have many annotated open-source data, I ...
- 83
0
votes
2
answers
50
views
Why shouldn't we try to balance the test set?
Most advice I have found online is that we must not balance the test set. The test set should remain to be unseen.
However, I failed to see how balancing the test set will cause us to leak knowledge ...
- 119
0
votes
1
answer
80
views
Multilabel classification: Choosing threshold
I'm creating a multilabel classification approach based on sentence embeddings applied to text taken from a chatbot.
We have the following:
a training dataset of 2,500 lines, where each line is a ...
0
votes
1
answer
28
views
Is it possible to train a Support Vector Machine to a specific accuracy?
From my understanding, support vector machines run on the premise of minimizing some error function, usually with the goal of maximizing accuracy overall. However, there are a lot of contexts, ...
- 27
0
votes
1
answer
40
views
Why an already trained model is not generalizable to another related dataset?
A model is trained to predict the median temperature of Boston. The resulting model works well according to their validation data. However, this model performs poorly when used to predict the ...
- 1,297
0
votes
1
answer
54
views
Why does the application of the logarithmic function improve the outcome of Random forests?
I have a Random forest model that tries to predict what kind of a useful activity a machine is doing based on its power readings. There are 5 features in a single reading.
There are two main types of ...
- 21
0
votes
0
answers
27
views
What if many different models reach the same maximum metrics
I am talking about trying different algorithms, different parameters, different stacking configurations that all improve upon previous baselines, and yet a lot of them have exactly the same values on ...
- 636
1
vote
1
answer
40
views
How does batch normalization make a model less sensitive to hyperparameter tuning?
Question 22 of 100+ Data Science Interview Questions and Answers for 2022 asks What is the benefit of batch normalization?
The first bullet of the answers to this is The model is less sensitive to ...
- 172
0
votes
1
answer
41
views
About neural network ability to generalize
I had this question during an interview that I wasn't able to answer, even after researching on the internet.
Which of the following can affect an artificial neural network’s ability to generalize???
...
2
votes
1
answer
295
views
Which type of models generalize better, generative or discriminative models?
In NLP, which type of models (generative or discriminative) is more sensitive to the amount of data to generalize better? references?
This is related to the way those two types capture the data ...
- 252
0
votes
0
answers
26
views
Can there be scenarios where an overfitted model in machine learning cannot be generalized?
Is it always possible to generalize an overfitted model? I know there are ways to handle overfitting, but can there be scenarios where overfitting cannot be handled in machine learning?
- 149
1
vote
2
answers
587
views
Graph Neural Network fails at generalizing on unseen graph topologies
I'm using PytorchGeometric to train a graph convolutional network for regression over nodes problem (the graph models physical phenomena in the network of sensors; the network of sensors is actually ...
- 11
1
vote
0
answers
23
views
Generalization error problem on training set
Training data:
$\mathcal {T} =\{(2,1),(3,2),(4,6),(0,0),(1,1)\}$
you already computed a predictor for the output using linear regression by least squares, where you used the first 3 samples as ...
- 11
2
votes
1
answer
293
views
Does convergence equal learning in Deep Q-learning?
In my current research project I'm using the Deep Q-learning algorithm. The setup is as follows: I'm training the model (using Deep Q-learning) on a static dataset made up of experiences extracted ...
- 81
1
vote
3
answers
89
views
1
vote
0
answers
27
views
Reducing High generalization-error on industrial fault data
I have a industrial dataset containing labeled machine data for fault classification(3 classes: 1 ok, 2 for faults). The problem is that i have less (~16) different machines, thus iam currently having ...
- 41
9
votes
2
answers
2k
views
High accuracy on test-set, what could go wrong?
You are given a pre-trained binary ML classification model with 99% accuracy on the test-set (assume the customer required 95% and that the test-set is balanced). We would like to deploy our model in ...
- 193
2
votes
1
answer
306
views
Multilabel Classification - Overfitting?
My task is the following:
To input drug combinations and output renal failure-related symptoms from the drug combinations.
Both the drug combinations and renal-failure related symptoms are represented ...
- 1,408
-1
votes
1
answer
198
views
Why when my local cv of loss decreases, my leaderboard's loss increases?
I got a cv log_loss of 0.3025410331400577 when using 4-fold cross-validation and my leaderboard (with 30% of test dataset) got 0.26514.
I further did feature engineering and added some features to the ...
1
vote
1
answer
66
views
What is the form of data used for prediction with generalized stacking ensemble?
I am very confused as to how training data is split and on what data level 0 predictions are made when using generalized stacking. This question is similar to mine, but the answer is not sufficiently ...
- 309
1
vote
1
answer
177
views
Upper bound on size of sample set for decision trees
Say I have an instance space with 4 features and I know that a decision tree with 8 nodes can represent the target function I want to learn. I want to give an upper bound on the size of the sample set ...
- 141
0
votes
1
answer
1k
views
how to shuffle the data for model.fit with custom data generator?
So trainfiles is a list that contains the files' directory and name e.g. ['../train/1.npy' , '../train/2.npy']
and then I create a dataset as shown in the middle of the code
then I apply it to model ...
- 13
0
votes
2
answers
744
views
Compare Classification Performance in Datasets drawn from Different Populations
I've read some classics about comparison of ML Algorithms i.e.
...
- 69
2
votes
1
answer
334
views
Why does this paper say that 0-1 loss is insensitive to scaling of weights in a neural network?
When discussing capacity control using norms of weights in a neural network,this paper says the following(see P4):
Capacity control in terms of norm, when using a zero/one loss (i.e. counting ...
- 121
1
vote
2
answers
99
views
Will oversampling help with generalization (small imbalanced dataset)?
I have an imbalanced dataset (2:1 ratio) with about 60 patients and 80 features.
I performed Recursive Feature Elimination (RFE) and stratified cross validation to reduce the features to 15 and I get ...
- 31
0
votes
1
answer
70
views
Are there some research papers about text-to-set generation?
I have googled but find no results.
Text-to-(word)set generation or sequence-to-(token)set generation.
For example, input a text and then output the tags for this text:
...
- 681
5
votes
1
answer
48
views
On design of the training set: conceptual question
I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
- 521
1
vote
0
answers
20
views
How much can be learned from a dataset?
Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset?
I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
- 21
1
vote
1
answer
593
views
Generalization of RNN/LSTM/GRU... model
Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this:
to_predict (orange) was fed to the model, predicted (purple) is the ...
- 111
0
votes
1
answer
41
views
Is generalizing a model, then removing the generalization good for FFNNs?
If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
- 119
2
votes
1
answer
304
views
XGBoost: what to do when Kfold is not enough?
I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
- 23
0
votes
1
answer
213
views
How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?
Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
- 312
1
vote
0
answers
70
views
Network either overfits or underfits, but never generalizes - what to do?
I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
- 11
3
votes
1
answer
658
views
What is NLP technique to generalize manually created rules in text?
Let's say we have a free text containing key-value entities.
Example: "... patient's tumour has width 6 cm and height 5 cm"
Then an expert comes, marks it as important, thus we do have the rule for ...
- 133
5
votes
1
answer
1k
views
Training on accurate data versus noisy data
I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
- 185
0
votes
2
answers
211
views
Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?
I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data.
Was wondering ...
- 145
0
votes
2
answers
1k
views
Why is it bad to use the same test dataset over and over again?
I am following this Google's series: Machine Learning Crash Course.
On the chapter about generalisation, they make the following statement:
Good performance on the test set is a useful indicator of ...
1
vote
2
answers
303
views
How is the equation for the relation between prediction error, bias, and variance defined?
I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned:
If we denote the variable we are trying to predict as $Y$ and our
covariates as $X$, we may assume that there is a ...
- 392
12
votes
2
answers
4k
views
The differences between SVM and Logistic Regression
I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
- 6,031
5
votes
1
answer
2k
views
The connection between optimization and generalization
Optimization algorithms such as gradient descent or particle swarm can find a minima in a function.
On the other hand, learning methods such as back-prop define learning as an optimization problem ...
- 151