First, I made the matrices (x, 129) to have the same shape by padding in with the rows filled with zeros.
Second, I transformed my large matrices into dimension 3 by the following process:
train = list(train_df['spec'])
>>> (50395, 71, 129)
X_train = list(train['spec']); X_valid = list(valid['spec'])
y_train = train['label'...
You just need to make sure that your x and y values have the same row dimension. This can be done using numpy.transpose or numpy.swapaxes.
import numpy as np
t = np.random.rand(150528, 1235)
np.transpose(t).shape # (1235, 150528)
np.swapaxes(t, 0, 1).shape # (1235, 150528)
This should give the shapes of (1235, 150528) for x and (1235, 154457) for y.
For this problem, you need to develop three models:
Model 1- for two main categories
Model 2- for sub-category A
Model 3- For sub-category B
So when you want to predict the result for an unseen data, first you use the Model 1, to find the main category. Based on the prediction and by using an if-else statement, you decide to perform another prediction using ...
Change your code to this.
pred = model.predict(X_train)
print("Precision: %f "%precision_score(y_train, pred))
print("Recall: %f "%recall_score(y_train, pred))
print("F1: %f"% f1_score(y_train, pred))
The problem is you are training the model but you have not made predictions to compare them with true values. you first have to ...
The error is because the model is expecting input of shape of 4 dimensional data [None, 224, 224, 3] where None is the Batch size which can be anything. but you are giving a single image of shape which is three dimensional.
try model.predict([man]) here we are just changing dimension to 4D with a batch size of 1.
The problem was due to the output format. Instead of outputting one value per input sequence, in order to produce a complete and unique sequence as output, it was outputting sequences. Though instead of having a unique sequence as result it got a sequence of sequences. The solution was to fix the network output shape and data shape as well.
Yes, you can get a neural network to predict real numbers instead of for classification. This type of problem is called regression.
You can find dozens of tutorials for doing regression with Keras. The official one is here.
If done standalone, then it is correct.
But another goal while applying augmentation is to have randomness.
This is achieved with multiple augmentation techniques applied together.
In that sense, these two can also become effective.
e.g. This is a zoomed image, adding vertical shift can crop the image further and eventually result in a new(random) image
You are using the pred variable to calculate your metrics, which will not work since pred is history callback object. The predictions from the model are not stored during model training, only your model metrics (e.g. loss and accuracy) are stored using the history callback. If you want the predicted output for samples you can use the model.predict method. ...
Since CNNs are translation invariant, aren't the translational shifts
from keras useless as those shifts will not result in new images per
Invariance to translation means that if we translate the inputs the CNN will still be able to detect the class to which the input belongs. Translational Invariance is a result of the pooling operation. In pooling ...
Try printing the shapes of your input matrix to debug,
Somewhere you are expecting a int value but it is giving you None, But the model you have created is expecting an Int value hence it is throwing an error.
This is not a question for which a particular answer could be provided, but the above sollution will surely encapsulate the answer you need.
As it already has been said, to_categorical() is function. It in keras for tensorflow 2.x can be imported this way:
from keras.utils import to_categorical
then used like this:
it will print
[0. 0. 0. 0. 0. 0. 1. 0. 0. 0.]
Where 10 is the number of classes, the input values range is [0;number_of_classes-1]. The ...
The binary cross-entropy loss function is based on the assumption that there is only one output node and it can have a value between 0-1.
If you have more than two outputs and using the softmax activation function, you should use a categorical cross-entropy loss function in order to handle the multiclass situation.
But in your scenario, there is a binary ...
Let's take a TS data = [ 1, 2, 3, 4, 5, 7, 8, 9, 10 ]
Call the function with these parameters
sequence_length=5, sampling_rate=1, sequence_stride=1, shuffle=False, batch_size=2
shuffle, batch_size has no role in TS data creation. It will come into effect when you iterate on the returned Dataset.
In this case, we will have the following data points,
[ 1, 2, ...
It can’t find any classes because test has no subdirectories. Without classes it can’t load your images, as you see in the log output above. There is a workaround to this however, as you can specify the parent directory of the test directory and specify that you only want to load the test “class”:
datagen = ImageDataGenerator()
test_data = datagen....
You could try using the flow_from_directory() method on your ImageDataGenerator class, which does the augmentation - only a small change is necessary:
H = model.fit(
aug.flow_from_directory(trainX, trainY, batch_size=BS),
If you start using a tf.data.Dataset directly, you will get more control over how the data is read from disk (caching, ...
Your batch size is y_true.shape
To normalized, which I assume you are looking for loss per observations what you need is below,
def custom_loss(y_true, y_pred):
return K.sum(y_true, y_pred) / tf.constant(y_true.shape, dtype=tf.int32)
Or why not just take the mean?
def custom_loss(y_true, y_pred):
return K.mean(y_true, y_pred)
Assuming 2 partitions since, for index in range(0,2) and a fixed bunch size of 3 since, start_range + 3
The shape of your y_true should be (nb_samples, 2), where the first vector of 2nd dimension is the ground_truth and the other is partition_index
import numpy as np
import tensorflow as tf
import tensorflow.keras.backend as K
from tensorflow.keras.losses ...
When you divide by 255 in the prediction make sure that the datatype is float and not int or unsigned int.
Make sure to implicitly convert the image x to float and divide by 255.0 before prediction. In other words make sure that you array is not all 0. You can still use Sigmoid for binary classifica with 1 dense layer. It should work
Seems like converting to TensorRT improves performance by more than 10x (!) for me, which I didn't expect at all.
The downside is, loading the TensorRT model takes >2min now and for reasons I can't grasp the script eats 2.2G of memory. Also getting the conversion process to work was beyond painful. I'm gonna open a new Q&A on that topic, because it ...
You can try PyTorch. It offers much more manual controls and tweaking and it's pure python ie no functional API that's why it is used in research fields whereas Keras is most easy and robust. I use Keras with a backend as plaidml which enables me to train my neural network models on AMD GPU (RX 560x).
Using argmax with model.predict is the way to go.
# testdata is the dataframe of Generator
paths = testdata.filenames # Your files path
y_pred = model.predict(testdata).argmax(axis=1) # Predict prob and get Class Indices
classes = testdata.class_indices # Map of Indices to Class name
from keras.preprocessing import image
a_img_rand = np.random.randint(0,...
I was able to get to work like this without flask or django. Just using default http.server in python
from http.server import BaseHTTPRequestHandler, HTTPServer
import pandas as pd
from keras.models import load_model
from urllib.parse import urlparse
model = load_model('model.h5')
The easiest way I can think of is to create a flask app that will load the model once and have and endpoint where you can send your data as requests to your already loaded model.
A rough skeleton of the service would look like this:
from flask import Flask, request
app = Flask(__name__)
If you choose metrics=['accuracy'], Keras automatically infers the accuracy metric according to the loss function. Four your case, since the loss function is BinaryCrossentropy, Keras has already chosen the metrics=['BinaryAccuracy'].
It can be shown that any function can be approximated using multi-layer networks that are fully connected and have nonlinear activations. In your case, if you add one more fully connected layer other than the current ones, you can achieve a better outcome. People usually add two hidden fully connected layers after convolutional layers and before the output ...
You don't actually need to apply the class labels, these don't matter. Keras will detect these automatically for you. It does this by studying the directory your data is in. Make sure you point to the parent folder where all your data should be. Your data should be in the following format:
It could be a number of different reasons but when I had that problem in the past it was usually due to too high of a learning rate or the optimizer. I recommended either dropping the initial learning rate or going with vanilla SGD. Occasionally I saw problems with Adam particularly if you have no warmup. You might want to try more general hyperparameter ...
I managed to obtain better results with less artifacts by implementing the code of the paper "High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks".
I found this code to be quite nice working for my purposes.
If you need, I can also provide a Colab notebook with my ported code.
A dense layer will output a fixed-sized vector. This will be what you want for a classification task for example, say sentiment classification.
A TimeDistributedDense will apply a dense layer to each output of the sequence. So let's say you have a text input, represented as a sequence of word embeddings, you would apply an LSTM cell and then the same dense ...
I don't know it this is an optimal approach. The natural way to solve a SUDOKU with computer science is Linear Programming. I am curious if a CNN will solve the problem.
Can you add +1 or -1 in the post processing / pre processing of the predictions? It will be the easier way to solve it.
The tough part about this problem is evaluating what moves are "correct" per se. In a fighting game sequence, there may be 2 or more moves which both work in theory in the current frame. If you are optimizing for knocking the other character out of frame, it would be useful to build a reward set which optimizes for this.
It may be worthwhile to ...
Let's clear it :
Assume you have a dataset with 8000 samples (rows of data) and you choose a batch_size = 32 and epochs = 25
This means that the dataset will be divided into (8000/32) = 250 batches, having 32 samples/rows in each batch. The model weights will be updated after each batch.
one epoch will train 250 batches or 250 updations to the model.
The real problem is that you should not try to fit all your images in memory. Instead, you should small groups of images, normally called "minibatches", which can fit in the GPU/CPU memory.
For that, tensorflow offers the function tf.keras.preprocessing.image_dataset_from_directory that loads images from a directory. I suggest you take a look at ...
You can simply train your convolutional layers, save the model and load weights from specific layers using the get_weights and set_weights methods (see also this previous answer). After loading the weights for you convolutional layers you can freeze those layers using the trainable attribute to make sure the weights are not changed during training.
It has to be saved somewhere i.e. Database after the training is done.
The saved values should be used on new data and all these steps should work in a loop i.e. when you re-train again the values will be updated and saved again.
e.g. If we see the Keras pre-trained models, it provides the necessary pre-processing function. We can directly use that