Keras Conv1D for simple data target prediction

Question

I am trying to use conv1D layer from Keras for predicting Species in iris dataset (which has 4 numeric features and one categorical target). Following is my code:

import numpy as np
import pandas as pd

irisdf = pd.read_csv('iris.csv')

Xall = irisdf.drop('Species', axis=1)
print(Xall.shape)
Xall = np.expand_dims(Xall.values, axis=2) 
print(Xall.shape)

Yall = irisdf['Species']
nb_classes =  3

import keras
from keras.models import Sequential
from keras.layers import Dense, InputLayer, Dropout, Flatten, BatchNormalization, Conv1D
input_shape = (Xall.shape[1:],)
print(input_shape)
model = Sequential([
    InputLayer(input_shape=input_shape), 
    Conv1D(32, 2),
    Dense(nb_classes, activation='softmax')
])
model.compile(loss=keras.losses.mean_squared_error,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])
model.summary()
model.fit(Xall, Yall, epochs=25, verbose=True)

However, it is giving following error:

Traceback (most recent call last):
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/eager/execute.py", line 141, in make_shape
    shape = tensor_shape.as_shape(v)
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 946, in as_shape
    return TensorShape(shape)
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 541, in __init__
    self._dims = [as_dimension(d) for d in dims_iter]
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 541, in <listcomp>
    self._dims = [as_dimension(d) for d in dims_iter]
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 482, in as_dimension
    return Dimension(value)
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_shape.py", line 37, in __init__
    self._value = int(value)
TypeError: int() argument must be a string, a bytes-like object or a number, not 'tuple'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "rnkeras_conv1d_iris.py", line 40, in <module>
    InputLayer(input_shape=input_shape), 
  File "/home/abcde/.local/lib/python3.5/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
    return func(*args, **kwargs)
  File "/home/abcde/.local/lib/python3.5/site-packages/keras/engine/input_layer.py", line 86, in __init__
    name=self.name)
  File "/home/abcde/.local/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py", line 515, in placeholder
    x = tf.placeholder(dtype, shape=shape, name=name)
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/ops/array_ops.py", line 1735, in placeholder
    return gen_array_ops.placeholder(dtype=dtype, shape=shape, name=name)
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4923, in placeholder
    shape = _execute.make_shape(shape, "shape")
  File "/home/abcde/.local/lib/python3.5/site-packages/tensorflow/python/eager/execute.py", line 143, in make_shape
    raise TypeError("Error converting %s to a TensorShape: %s." % (arg_name, e))
TypeError: Error converting shape to a TensorShape: int() argument must be a string, a bytes-like object or a number, not 'tuple'.

Where is the problem and how can it be solved?

(PS: If you find this question to be interesting/important, please upvote it;)

Answer 1

Your error is coming from the Keras framework not working with strings as the output labels. You will want to transform these to 1-hot encoded vectors to train your model. Here is some code to do this.

Getting the data

import pandas as pd
df = pd.read_csv('iris.csv', header=None, names=['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species'])

This will assign a class label, we will one-hot encode them later

df['labels'] =df['species'].astype('category').cat.codes

Splitting the data and reshaping the data

First we will split the data into a training and testing set. Then we will one-hot encode the labels. And finally we will structure the inputs to match what is expected from Keras. To use a 1D convolution we need to add a spatial dimension.

from sklearn.model_selection import train_test_split
import keras

X = df[['sepal_length', 'sepal_width', 'petal_length', 'petal_width']]
Y = df['labels']
x_train, x_test, y_train, y_test = train_test_split(np.asarray(X), np.asarray(Y), test_size=0.33, shuffle= True)

# The known number of output classes.
num_classes = 3

# Input image dimensions
input_shape = (4,)

# Convert class vectors to binary class matrices. This uses 1 hot encoding.
y_train_binary = keras.utils.to_categorical(y_train, num_classes)
y_test_binary = keras.utils.to_categorical(y_test, num_classes)

x_train = x_train.reshape(100, 4,1)
x_test = x_test.reshape(50, 4,1)

The model

Your model was insufficient to get good results so I added an additional hidden layer into the mix to get acceptable results.

from __future__ import print_function    
from keras.models import Sequential
import keras
from keras.models import Sequential
from keras.layers import Dense, Flatten, Conv1D
from keras.callbacks import ModelCheckpoint
from keras.models import model_from_json
from keras import backend as K

model = Sequential()
model.add(Conv1D(32, (3), input_shape=(1,4), activation='relu'))
model.add(Flatten())
model.add(Dense(64, activation='relu'))
model.add(Dense(num_classes, activation='softmax'))

model.compile(loss=keras.losses.categorical_crossentropy,
              optimizer=keras.optimizers.Adadelta(),
              metrics=['accuracy'])

model.summary()

Now let's train the model

batch_size = 128
epochs = 10
model = model.fit(x_train, y_train_binary,
          batch_size=batch_size,
          epochs=epochs,
          verbose=1,
          validation_data=(x_test, y_test_binary))

100/100 [==============================] - 0s 50us/step - loss: 1.0906 - acc: 0.6400 - val_loss: 1.0893 - val_acc: 0.7000

We get 70% accuracy, that's not so bad. But it can be improved by changing the model to better suit the data source.

Plot the convergence

plt.plot(model.history['loss'])
plt.plot(model.history['val_loss'])
plt.title('model train vs validation loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'validation'], loc='upper right')
plt.show()