We’re rewarding the question askers & reputations are being recalculated! Read more.

Questions tagged [generalization]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0
votes
0answers
16 views

Learning discrete probability distribution that is parametrized by a set of real-valued parameters

Assume I have a discrete probability distribution defined over binary variables. This probability distribution is parametrized by a set of real-valued parameters, which all are contained in a segment, ...
5
votes
1answer
29 views

On design of the training set: conceptual question

I am curious to know how training data should be constructed so that it scales to examples that are not a part of the training data. For example, the problem that I am facing right now is in the ...
0
votes
0answers
7 views

How is possible the result of GRU would other way around compared to LSTM while they were implemented samely?

Recently I crossed to a situation I can't figure it out why it happened. I applied six predictive models on a specific dataset as training-set and tried to predict the other similar dataset as an ...
1
vote
0answers
15 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
1
vote
0answers
61 views

Generalization of RNN/LSTM/GRU… model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
0
votes
1answer
15 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
0
votes
0answers
20 views

Is a small but constant learning rate better for generalization than learning rate decay?

Learning rate decay - starting with a higher learning rate for fast convergence and then decreasing the learning rate for better convergence - allows training loss to converge to the same value in ...
0
votes
0answers
44 views

What to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
0
votes
1answer
75 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...
1
vote
0answers
57 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
1
vote
1answer
207 views

What is NLP technique to generalize manually created rules in text?

Let's say we have a free text containing key-value entities. Example: "... patient's tumour has width 6 cm and height 5 cm" Then an expert comes, marks it as important, thus we do have the rule for ...
4
votes
1answer
714 views

Training on accurate data versus noisy data

I have data currently available that is very accurate and I would like to train my classification methods on this set of clean data to learn the important markers for distinguishing between classes. ...
0
votes
2answers
88 views

Is a Neural Network with 20 times the number of input neurons (on hidden layers) guaranteed to overfit? When is this not so?

I'm aware of the problem of over-fitting. One way to describe it is your Neural Network learning your training data to a high accuracy and performing poorly (generalizing) on new data. Was wondering ...
0
votes
2answers
30 views

Why is it bad to use the same test dataset over and over again?

I am following this Google's series: Machine Learning Crash Course. On the chapter about generalisation, they make the following statement: Good performance on the test set is a useful indicator of ...
1
vote
2answers
211 views

How is the equation for the relation between prediction error, bias, and variance defined?

I'm reading this article Understanding the BiasVariance Tradeoff. It mentioned: If we denote the variable we are trying to predict as $Y$ and our covariates as $X$, we may assume that there is a ...
8
votes
2answers
474 views

The differences between SVM and Logistic Regression

I am reading about SVM and I've faced to the point that non-kernelized SVMs are nothing more than linear separators. Therefore, ...
4
votes
1answer
553 views

The connection between optimization and generalization

Optimization algorithms such as gradient descent or particle swarm can find a minima in a function. On the other hand, learning methods such as back-prop define learning as an optimization problem ...