Questions tagged [generalization]

The tag has no usage guidance.

8 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
1
vote
0answers
15 views

How much can be learned from a dataset?

Is there a way to quantify or characterize some upper limits for how much one can "learn" from a dataset? I got puzzled, when the neural state machine got me thinking components of AGI and lead me ...
1
vote
0answers
51 views

Generalization of RNN/LSTM/GRU… model

Given a time-series prediction with a Recurrent Neural Network (doesn't matter if LSTM/GRU/...), a forecast might look like this: to_predict (orange) was fed to the model, predicted (purple) is the ...
1
vote
0answers
57 views

Network either overfits or underfits, but never generalizes - what to do?

I have a simple network with 1st level an LSTM, dropout, fully-connected and softmax layers; loss is cross-entropy (four classes, well balanced). Sequence length to LSTM is 172 samples, data is z-...
0
votes
0answers
6 views

How is possible the result of GRU would other way around compared to LSTM while they were implemented samely?

Recently I crossed to a situation I can't figure it out why it happened. I applied six predictive models on a specific dataset as training-set and tried to predict the other similar dataset as an ...
0
votes
1answer
15 views

Is generalizing a model, then removing the generalization good for FFNNs?

If one is training a basic FFNN (Feed-Forward Neural Network), one would apply regularizations like dropout, l1, l2 and gaussian noise, so that the model is robust and gives better results for unseen ...
0
votes
0answers
19 views

Is a small but constant learning rate better for generalization than learning rate decay?

Learning rate decay - starting with a higher learning rate for fast convergence and then decreasing the learning rate for better convergence - allows training loss to converge to the same value in ...
0
votes
0answers
43 views

What to do when Kfold is not enough?

I have a dataset made of roughly 100 time-series and my final goal is to obtain a classification of each point (detection problem). To do so I have labels so I decided to use an XGB model to perform ...
0
votes
1answer
67 views

How can I measure the reliability of the specificity of a model with very small train, test, and validation datasets?

Stats newbie here. I have a small dataset of 646 samples that I've trained a reasonably performant model on (~99% test and val accuracy). To complicate things a little bit, the classes are somewhat ...