This means your model is overfitting the train set. Try simplifying the model by reducing the number of neurons in each layer. That will reduce your train accuracy but may allow your validation accuracy to increase.
You know that you are not overfitting when both train and validation accuracy hover around the same value.
Your model is probably overfitting. You could try following things (or mixture of these):
Remove layers or reduce the number of neurons
Use dropout technique
Use regularization (e.g L1,L2)
Use Data Augmentation
copied from: https://www.kdnuggets.com/2019/12/5-techniques-prevent-overfitting-neural-networks.html
First, note that they are just adding 1 to the size of the vocabulary, not to the token IDs themselves, so the predictions are not affected.
Then, why adding 1 ?
Because Tokenizer.word_index is a python dictionary that contains token keys (string) and token ID values (integer), and where the first token ID is 1 (not zero) and where the token IDs are assigned ...
[I've added this answer as I think others miss the main theoretical gist.]
Firstly, NCE and Negative Sampling (NS) serve different purposes:
NS is a generic trick used to train a classifier if you only have training samples from one `positive' class (e.g. labelled $y\!=\!1$);
NCE is a method to learn parameters $\theta$ of a model $p_m(x;\theta)$ of a true ...
We explain AI with intuition rather than maths most of the time, so everyone has its own explanation and representation of things, here is how I would explain activation functions (I'll try to make is clearer instead of adding another version to the already 2 you know):
Just in case you don't know what a basis is, you should have a look at Wikipedia since it ...
To get an output on every step, you have to make return_sequence=True for all LSTM layers
The last Dense layer should reflect the output size i.e. 3 here
Hence, the shape of Y = (total instances, n_steps, output size)
You may try Keras DepthwiseConv2D layer
Depthwise Separable convolutions consist of performing just the first step in a depthwise spatial convolution (which acts on each input channel separately). The depth_multiplier argument controls how many output channels are generated per input channel in the depthwise step.
It will convolute each Channel separately. ...
d_model is the dimensionality of the representations used as input to the multi-head attention, which is the same as the dimensionality of the output. In the case of normal transformers, d_model is the same size as the embedding size (i.e. 512). This naming convention comes from the original Transformer paper.
depth is d_model divided by the number of ...
It seems that you have cuda 10 installed, and not version 11 which is what TensorFlow is looking for. It has rather specific requirements to get working.
Tensorflow 2.4.1 requires cuda 11 and cudnn 8 (see GPU table of requirements here). I would suggest checking also, your nvidia driver version, 450.x or higher is required.
This is the TensorFlow official ...
You may try cycle GANs.
In normal Generative Adversarial Networks (GANs), a generative network -the generator- is trained to generate images with an auxiliary network, the discriminator. The discriminator learns to tell apart data generated by the generator (i.e. fake data) from real data, while the generator learns to generate data that fools the ...
The reason the tensor takes up so much memory is because by default the tensor will store the values with the type torch.float32. This data type will use 4kb for each value in the tensor (check using .element_size()), which will give a total of ~48GB after multiplying with the number of zero values in your tensor (4 * 2000 * 2000 * 3200 = 47.68GB). What you ...
One is a straight forward use of API while the other one gives you much better control on training. If you are experimenting, it is good to go with a custom training loop so you can control the various things that happen to your model's training.
But, if you want to just train a model and not experiment with it much, save time and go for the straight forward ...
Figured it out. Need to normalize the image I'm going to predict to $[-1,1]$ instead of what-I-assume-is-the-standard $[0,1]$. It seems this is model-dependent. The project I'm working off of uses transfer learning and imports this model.
It's not immediately clear to me though just how this model needs inputs normalized to $[-1,1]$.
Using model.fit() will always train on the whole dataset for n epochs. The batch_size argument denotes how many samples are used to calculate the gradient and updates the parameters. If you want to train the model on just a batch of data (i.e. a subset of your total dataset) simply pass the subset of data to model.fit().
Glad you found where it went wrong! However, it is really possible for something like that to happen. There is no such thing as "best algorithm", so the performance of a method partly depends on what your dataset looks like. Or sometimes your feature engineering method just allows the data to cheat on you, say, you mistakenly leaked some data, or ...
Yes, this could be possible if your dev/test data comes from the same domain as the training data, in which case word2vec will encounter fewer OOV tokens that mess up the loss.
This could also mean that the benefits of BERT - subword tokenization to handle OOV characters in generalized domains - are lost. If your vocabulary size is small, your word2vec model ...
This is overfitting, and it suggests that your images in each class are very similar to images across other classes.
Since your images across classes seem very similar, 800 per class is actually not a lot of data to train on. It's likely your model is struggling to discriminate the dev data into the correct classes based on what little it can learn., and ...