1
$\begingroup$

I am training a CNN, it runs well but I get an error when I try to get precision_score, recall_score and the f1_score. Here is a snippet of my code;

# The next step is to split training and testing data. For this we will use sklearn function train_test_split().
features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=.2)

features_train.shape, features_test.shape, labels_train.shape, labels_test.shape

((180568, 2677356), (45143, 2677356), (180568,), (45143,))

features_train.shape[0], features_train.shape[1], labels_train.shape[0]

(180568, 2677356, 180568)

n_timesteps, n_features, n_outputs = features_train.shape[0], features_train.shape[1], labels_train.shape[0]

X_train = np.zeros((180568, 82, 1))
y_train = np.zeros((180568, 82))
n_timesteps, n_features, n_outputs = X_train.shape[1], X_train.shape[2], y_train.shape[1]
n_samples = 1000

X = np.random.uniform(0,1, (n_samples, n_timesteps, n_features))
y = pd.get_dummies(np.random.randint(0,n_outputs, n_samples)).values

model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=32),
tf.keras.layers.MaxPooling1D(strides=3),
#tf.nn.local_response_normalization((1, 1, 1, 1), depth_radius=5, bias=1, alpha=1, beta=0.5, name=None),
tf.keras.layers.LayerNormalization(axis=1),
tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=64),
tf.keras.layers.MaxPooling1D(strides=3), # also GlobalMaxPooling1D() is ok
tf.keras.layers.LayerNormalization(axis=1),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(n_outputs, activation='softmax')
]) 
model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])

model.summary()
Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d (Conv1D)              (None, 81, 32)            96        
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 27, 32)            0         
_________________________________________________________________
layer_normalization (LayerNo (None, 27, 32)            54        
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 26, 64)            4160      
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 9, 64)             0         
_________________________________________________________________
layer_normalization_1 (Layer (None, 9, 64)             18        
_________________________________________________________________
flatten (Flatten)            (None, 576)               0         
_________________________________________________________________
dense (Dense)                (None, 82)                47314     
=================================================================
Total params: 51,642
Trainable params: 51,642
Non-trainable params: 0
history = model.fit(X_train, y_train, epochs=10, verbose=1, validation_data=(X, y))
Train on 180568 samples, validate on 1000 samples
Epoch 1/10
180568/180568 [==============================] - 130s 718us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 2/10
180568/180568 [==============================] - 128s 707us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 3/10
180568/180568 [==============================] - 101s 561us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 4/10
180568/180568 [==============================] - 77s 426us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 5/10
180568/180568 [==============================] - 75s 413us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 6/10
180568/180568 [==============================] - 75s 418us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 7/10
180568/180568 [==============================] - 113s 624us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 8/10
180568/180568 [==============================] - 109s 602us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 9/10
180568/180568 [==============================] - 115s 639us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0693 - val_acc: 0.9878
Epoch 10/10
180568/180568 [==============================] - 129s 716us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0693 - val_acc: 0.9878
from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
print("Precision: %f "%precision_score(y_train, pred)) <---- This is where I get the error
print("Recall: %f "%recall_score(y_train, pred))
print("F1: %f"% f1_score(y_train, history))

The error I got is;

TypeError: Expected sequence or array-like, got <class 'tensorflow.python.keras.callbacks.History'>
$\endgroup$
0
$\begingroup$

You are using the pred variable to calculate your metrics, which will not work since pred is history callback object. The predictions from the model are not stored during model training, only your model metrics (e.g. loss and accuracy) are stored using the history callback. If you want the predicted output for samples you can use the model.predict method. This will return a numpy array of the predicted output from your model, which you can then use in your metric functions.

$\endgroup$
8
  • $\begingroup$ y_train are labels values while the pred values are is where I fit my model using the y_train values $\endgroup$ – humblefool_9 Jan 22 at 13:28
  • $\begingroup$ What do you get when you print the types of both your y_train and pred variables? $\endgroup$ – Oxbowerce Jan 22 at 13:38
  • $\begingroup$ y_train I get an array of zeros while for pred i get <tensorflow.python.keras.callbacks.History object at 0x7f0f8c303cd0> $\endgroup$ – humblefool_9 Jan 22 at 13:51
  • $\begingroup$ So then the errors comes from how you are assigning your pred variable, as it is not an array/list of values. Can you add code to your OP to show how you are creating the pred variable? $\endgroup$ – Oxbowerce Jan 22 at 13:55
  • $\begingroup$ the pred values should be the history in which am training my model if I do understand well or do I need to create that separately? $\endgroup$ – humblefool_9 Jan 22 at 14:03
0
$\begingroup$

Change your code to this.

pred = model.predict(X_train)
print("Precision: %f "%precision_score(y_train, pred)) 
print("Recall: %f "%recall_score(y_train, pred))
print("F1: %f"% f1_score(y_train, pred))

The problem is you are training the model but you have not made predictions to compare them with true values. you first have to make predictions using model.predict() method and then you can compare pred and y_train.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.