TypeError: Expected sequence or array-like, got <class 'tensorflow.python.keras.callbacks.History'>

Question

I am training a CNN, it runs well but I get an error when I try to get precision_score, recall_score and the f1_score. Here is a snippet of my code;

# The next step is to split training and testing data. For this we will use sklearn function train_test_split().
features_train, features_test, labels_train, labels_test = train_test_split(features, labels, test_size=.2)

features_train.shape, features_test.shape, labels_train.shape, labels_test.shape

((180568, 2677356), (45143, 2677356), (180568,), (45143,))

features_train.shape[0], features_train.shape[1], labels_train.shape[0]

(180568, 2677356, 180568)

n_timesteps, n_features, n_outputs = features_train.shape[0], features_train.shape[1], labels_train.shape[0]

X_train = np.zeros((180568, 82, 1))
y_train = np.zeros((180568, 82))
n_timesteps, n_features, n_outputs = X_train.shape[1], X_train.shape[2], y_train.shape[1]
n_samples = 1000

X = np.random.uniform(0,1, (n_samples, n_timesteps, n_features))
y = pd.get_dummies(np.random.randint(0,n_outputs, n_samples)).values

model = tf.keras.models.Sequential([
tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=32),
tf.keras.layers.MaxPooling1D(strides=3),
#tf.nn.local_response_normalization((1, 1, 1, 1), depth_radius=5, bias=1, alpha=1, beta=0.5, name=None),
tf.keras.layers.LayerNormalization(axis=1),
tf.keras.layers.Conv1D(input_shape=(n_timesteps, n_features), activation='relu', kernel_size=2, filters=64),
tf.keras.layers.MaxPooling1D(strides=3), # also GlobalMaxPooling1D() is ok
tf.keras.layers.LayerNormalization(axis=1),
tf.keras.layers.Flatten(),
tf.keras.layers.Dense(n_outputs, activation='softmax')
]) 

model.compile('adam', 'binary_crossentropy', metrics=['accuracy'])

model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d (Conv1D)              (None, 81, 32)            96        
_________________________________________________________________
max_pooling1d (MaxPooling1D) (None, 27, 32)            0         
_________________________________________________________________
layer_normalization (LayerNo (None, 27, 32)            54        
_________________________________________________________________
conv1d_1 (Conv1D)            (None, 26, 64)            4160      
_________________________________________________________________
max_pooling1d_1 (MaxPooling1 (None, 9, 64)             0         
_________________________________________________________________
layer_normalization_1 (Layer (None, 9, 64)             18        
_________________________________________________________________
flatten (Flatten)            (None, 576)               0         
_________________________________________________________________
dense (Dense)                (None, 82)                47314     
=================================================================
Total params: 51,642
Trainable params: 51,642
Non-trainable params: 0

history = model.fit(X_train, y_train, epochs=10, verbose=1, validation_data=(X, y))

Train on 180568 samples, validate on 1000 samples
Epoch 1/10
180568/180568 [==============================] - 130s 718us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 2/10
180568/180568 [==============================] - 128s 707us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 3/10
180568/180568 [==============================] - 101s 561us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0695 - val_acc: 0.9878
Epoch 4/10
180568/180568 [==============================] - 77s 426us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 5/10
180568/180568 [==============================] - 75s 413us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 6/10
180568/180568 [==============================] - 75s 418us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 7/10
180568/180568 [==============================] - 113s 624us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 8/10
180568/180568 [==============================] - 109s 602us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0694 - val_acc: 0.9878
Epoch 9/10
180568/180568 [==============================] - 115s 639us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0693 - val_acc: 0.9878
Epoch 10/10
180568/180568 [==============================] - 129s 716us/sample - loss: 0.0123 - acc: 1.0000 - val_loss: 0.0693 - val_acc: 0.9878

from sklearn.metrics import precision_score
from sklearn.metrics import recall_score
from sklearn.metrics import f1_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report

print("Precision: %f "%precision_score(y_train, pred)) <---- This is where I get the error
print("Recall: %f "%recall_score(y_train, pred))
print("F1: %f"% f1_score(y_train, history))

The error I got is;

TypeError: Expected sequence or array-like, got <class 'tensorflow.python.keras.callbacks.History'>

Answer 1

You are using the pred variable to calculate your metrics, which will not work since pred is history callback object. The predictions from the model are not stored during model training, only your model metrics (e.g. loss and accuracy) are stored using the history callback. If you want the predicted output for samples you can use the model.predict method. This will return a numpy array of the predicted output from your model, which you can then use in your metric functions.

Answer 2

Change your code to this.

pred = model.predict(X_train)
print("Precision: %f "%precision_score(y_train, pred)) 
print("Recall: %f "%recall_score(y_train, pred))
print("F1: %f"% f1_score(y_train, pred))

The problem is you are training the model but you have not made predictions to compare them with true values. you first have to make predictions using model.predict() method and then you can compare pred and y_train.