It might be worth switching to half precision floats which will reduce the memory use:
from tensorflow.keras import mixed_precision
policy = mixed_precision.Policy('mixed_float16')
And your last layer should be instead:
outputs = Conv3D(1, (1, 1, 1)) (c7)
outputs = layers.Activation('sigmoid', dtype='float32')(...
The shapes of your inputs/labels have been set up incorrectly which I am guessing is resulting in something funky happening in the calculations of loss.
You are setting input_shape=(None, None, 1) but your x_train has an input shape of (20,1,1,1). Firstly, these should have the same dimensions, with each None in the input shape indicating a variable batch ...
...Because the Keras documentation does not specify the keys for the class_weights..
You may get an idea with these two parameters,
labels: Either "inferred" (labels are generated from the directory structure), or a list/tuple of integer labels of the same size as the number of image files found in the directory. Labels should be sorted according ...
Glad you found where it went wrong! However, it is really possible for something like that to happen. There is no such thing as "best algorithm", so the performance of a method partly depends on what your dataset looks like. Or sometimes your feature engineering method just allows the data to cheat on you, say, you mistakenly leaked some data, or ...
A couple of things that jump out immediately as being a little odd:
You seem to have two labels "ball_count" and "run_per_ball" but only 1 output of the network.
You are using an MSE loss, which is typically used for regression problems. This seems to be a sensible choice for outputs like ball_count or run_per_ball. However, accuracy, is ...
There is another option with which you don't have to copy the test file into another test file:
datagen = ImageDataGenerator()
test_data = datagen.flow_from_directory('.', classes=['test'])
This solved my problem. For more info, see
Everything seems correct to my non-expert eyes in your code, exept lr in the code you shared (0.00001) seems low for a SGD, but as you mentionned, you tried different rates, so this probably is not the issue there.
I'm not familiar with elu or selu activation functions and usually use relu in all my layers, yet i can't say if that is the problem.
May you ...
Using model.fit() will always train on the whole dataset for n epochs. The batch_size argument denotes how many samples are used to calculate the gradient and updates the parameters. If you want to train the model on just a batch of data (i.e. a subset of your total dataset) simply pass the subset of data to model.fit().
Thanks to everyone who gave answer and comments. It was indeed caused by my data.
Prior to this I had the same preprocessing pipeline for both models, which would be your "usual" NLP preprocessing steps (non-alphanumerical removal, lowercasing, stemming, and stop word removal). I had a hunch that both stemming and stop word removal would cause the ...
This is overfitting, and it suggests that your images in each class are very similar to images across other classes.
Since your images across classes seem very similar, 800 per class is actually not a lot of data to train on. It's likely your model is struggling to discriminate the dev data into the correct classes based on what little it can learn., and ...
Yes, this could be possible if your dev/test data comes from the same domain as the training data, in which case word2vec will encounter fewer OOV tokens that mess up the loss.
This could also mean that the benefits of BERT - subword tokenization to handle OOV characters in generalized domains - are lost. If your vocabulary size is small, your word2vec model ...
It's hard to say for sure without knowing more about the size of your dataset, the architecture of your model, and the libraries you are using.
Changing from pandas.Dataframe to numpy.array is very unlikely to make your training faster. Pandas dataframes are already backed by numpy arrays.
Depending on what neural network library you are using and what ...
If tf dataset is used you cannot use the class_weights parameter. Insted return the weight from a parse_function in your pipeline
weight_arr = [1.5, 0.5] #define your custom weights
#create a lookup table
key_tensor = tf.constant(list(range(0, len(weight_arr))), dtype=tf.int64)
val_tensor = tf.constant(weight_arr)
init = tf.lookup....
I finally was able to solve my issue. It's weird but even though the custom multicategory layer did not have params it contained its own mapping for the data. To extend the model and to examine the effect of layer depths I created a new model by adding the multicategory layer from the existing model. Once I did this training accuracy matched AutoKeras.
It is likely that your train variable in kf.split(train): is a list of two lists e.g. train_x and train_y or something similar. I am guessing this because the KFold API is only detecting only two entries in it, which it is unable to partition in 5 subsets (folds).
What you are trying to address is a problem of hierarchical classification in contrast to the flat classification which we are very familiar with.
Some work has already been done to address such problems and it has been shown that single unified model outperforms layered architecure of multiple flat classifiers for individual tasks (e.g. in your case ...
The answer can be found by just printing
Layer (type) Output Shape Param #
dense (Dense) (None, 32) 160 ...
It depends on the problem. A shape of [1000,1], suggests that you are trying to predict a single label for each sequence member of the batch, each member having up to 10 tokens in the sequence. This could happen say, if you want to classify sentences of 10 words each as positive or negative sentiment, or basically if you have a label to apply to each ...
training's loss/accuracy are not calculated for the whole dataset every time.
It is calculated for the current batch and averaged successively.
test's loss/accuracy is calculated at the end of an epoch for the whole test data.
Check these references
You may put the validation data same as the train data to check it.
To me it looks like a clear case of overfitting and perhaps the main reason is that your model is far too complex for the problem. In order to differentiate beween over and under fitting you can think about learning in the following way.
The data which is given as example (training data) contains the bahavior of the true model with some additional noise. You ...
You are missing one argument in your code. Following will fix your problem.
After taking a look at your code, it seems that you've not employed any kind of regularization. You may want to use dropout. Moreover, in convolutional autoencoders, in the decoder part, there is a well-known artifact called checkerboard. I don't know how this can be a problem for your task since you're using one-dimensional convolution in the decoder. By ...
As rightly pointed out by you the rescale=1./255 will convert the pixels in range [0,255] to range [0,1].
This process is also called Normalizing the input. Scaling every images to the same range [0,1] will make images contributes more evenly to the total loss.
Without scaling, the high pixel range images will have a large say to determine how to update ...
You should reshape your numerical layers so that they have a shape (None,4,4,1). To concatenate, you need the all but one axis to be equal.
If you use:
this should work.
When you are using a pre-trained model, you should use it's specific pre-processing function,
Below is an example for resnet50.
from keras.applications.resnet50 import preprocess_input
train_datagen = ImageDataGenerator(#rescale = 1./255
Always display/print your image/label for a ...
You should look at the data with Features intact for each step.Featues can't be flattened since each point of time is defined by all the Features.
Let's see this snap,
The upper table is the data
Let's assume we want to predict 2 steps using 3 input steps.
So, one instance of our input will have 3 sequential steps having 6 Features part of each sequential ...
take a look at the paper "Generating Sentences from a Continuous Space" by Bowman. In Section 3.1 it is explained why LSTM_VAE tend to this behaviour:
"This problematic tendency in learning is compounded by the lstm decoder’s sensitivity to subtle variation in the hidden states, such as that introduced by the posterior sampling process. This
When layers are "frozen" it generally means that their weights are not updated when backpropagation happens. So, technically, there is no difference between:
Using a "frozen" VGG16 and training some fully connected layers and
Using the VGG16 embeddings and training some fully connected layers
In practice, if you did both and compared ...
I create two new objects in call(): layers.Flatten and layers.Dense. When I create the objects in the constructor, the model works as expected.
def __init__(self, channel):
self.cnn1 = CNNBlock(channel)
self.cnn2 = CNNBlock(channel)
self.flat = layers....
There are basically two parts.
Why there is a comma in the [3,]? In this case you can skip it and just use . You can encounter it in the tutorials, because you can pass the tuple as shape as well - (3,) and if you skip the comma in the tuple, then it will be just number, not tuple. So, it's just more of a python, not a keras itself. Try this in terminal
You are almost right. However, in your specific examples (None,) and (None,12), we actually do know the size of the model's input and that's why the model can be compiled. (None,) refers to scalar inputs and (None,12) refers to 12-dimensional input vectors.
Therefore, one can think of None as an adjustable variable/placeholder for the batch size of your ...
No you should declare it ahead of the class, just after importing the packages. You have declare the seed for numpy and tensorflow separately. For further details read this blog -