I am currently learning binary classification. The problem is classifying positive and negative movie reviews.
The dataset is 25,000 reviews with each review represented by 10,000 of the most used words. each review is transformed into multi-hot encoding and the labels are 1s and 0s for positive and negative reviews respectively.
This is the data preparation code:
from tensorflow.keras.datasets import imdb
(train_data, train_labels), (test_data, test_labels) = imdb.load_data(num_words=10000)
import numpy as np
def vectorize_sequences(sequences, dimension=10000):
results = np.zeros((len(sequences), dimension))
for i, sequence in enumerate(sequences):
for j in sequence:
results[i, j] = 1
return results
x_train = vectorize_sequences(train_data)
x_test = vectorize_sequences(test_data)
y_train = np.asarray(train_labels).astype("float32")
y_test = np.asarray(test_labels).astype("float32")
x_val = x_train[:10000]
partial_x_train = x_train[10000:]
y_val = y_train[:10000]
partial_y_train = y_train[10000:]
Now this is the initial model and evaluation:
from tensorflow import keras
from tensorflow.keras import layers
model = keras.Sequential([
layers.Dense(16, activation="relu"),
layers.Dense(16, activation="relu"),
layers.Dense(1, activation="sigmoid")
])
model.compile(optimizer="rmsprop",
loss="binary_crossentropy",
metrics=["accuracy"])
history = model.fit(partial_x_train,
partial_y_train,
epochs=20,
batch_size=512,
validation_data=(x_val,y_val))
results = model.evaluate(x_test,y_test)
With this model, if I only run 4 epochs I get the best loss and accuracy on the validation data and therefore on the test - accuracy 88%.
Now when I try different variations of the model architecture For example: different number of layers, number of units, using MSE loss instead of binary_crossentropy loss, using tanh instead of relu.
I don't get significant differences between all the models except sometimes different epochs are have the best loss and accuracy on the validation but the values of the best loss and accuracy doesn't change much.
Am I missing something or the different architectures really don't impact the predictions very much?