Classifying the sum of two inputs as even or odd - Model always stuck at 50% accuracy for training and test data

Question

I am creating a simple feed forward to classify if the sum of two inputs as even or odd.

I cannot change the input structure (has to be two nodes), and output structure (two nodes as well, one for even, one for odd).

Here is my code which can be run out of the box if you have PyTorch installed:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import time
from datetime import datetime
import pandas_datareader.data as web
from math import floor
from random import randrange
from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout
from keras.utils import to_categorical
from keras import optimizers
from keras import regularizers

# CONFIG 
input_sz = 2
output_sz = 2
split_pct = 0.1
iterations = 10

LEARNING_RATE = 0.001

hidden_layers = [8,8,8]  # Configurable

# END CONFIG

def sigmoid(x):
    return 1 / (1 + np.exp(-x))
    
data = []
for i in range(0, 100000):
    inputNum1 = randrange(100)
    inputNum2 = randrange(100)
    inp = [inputNum1/100, inputNum2/100]
    if (inputNum1 + inputNum2 ) % 2 == 0:
        out = [1]
    else:
        out = [0]
    data.append([inp, out])

train_data = data[:int(split_pct * len(data))]
test_data = data[int(split_pct * len(data)):]


# Build the model
model = Sequential()
for i, layer_size in enumerate(hidden_layers):
    if i == 0:
        model.add(Dense(layer_size, input_dim=input_sz, activation='relu')) # , kernel_regularizer=regularizers.l2(0.01)))
    else:
        model.add(Dense(layer_size, activation='relu')) #, kernel_regularizer=regularizers.l2(0.01)))
model.add(Dense(output_sz, activation='sigmoid'))

optimizer = optimizers.RMSprop(learning_rate=LEARNING_RATE)
model.compile(loss='binary_crossentropy', optimizer=optimizer, metrics=['accuracy'])

# Train the model
X_train = np.array([item[0] for item in train_data])
y_train = np.array([item[1] for item in train_data])

y_train = to_categorical(y_train, num_classes=output_sz)


while (True):

    model.fit(X_train, y_train, epochs=iterations, batch_size=32, verbose=1)

    # Test the model
    X_test = np.array([item[0] for item in test_data])
    y_test = np.array([item[1] for item in test_data])
    y_test = to_categorical(y_test, num_classes=output_sz)

    score = model.evaluate(X_test, y_test, verbose=0)
    print('Test loss:', score[0])
    print('Test accuracy:', score[1])

    #input("Press Enter to continue...")

Does anyone have any recommendations to make this model train properly? Thank you

Note: I've tried RMSProp, Adam, etc - same result, stuck at 50%. Also tried increasing/decreasing the learning rate and neurons/layers, and unfortunately no result either.

Answer 1

I see that you've decided to represent your input as:

inp = [inputNum1/100, inputNum2/100]

Why divide by 100? If you're manually transforming your input data, you have a much better choice of representation that you can choose to help your model learn by essentially turning this into an XOR problem. Also another thing, you have:

if i == 0:
        model.add(Dense(layer_size, input_dim=input_sz, activation='relu')) # , kernel_regularizer=regularizers.l2(0.01)))
    else:
        model.add(Dense(layer_size, activation='relu')) #, kernel_regularizer=regularizers.l2(0.01)))

I suggest you go back and look at why you chose ReLU as your activation function here. Does it make sense in the context of trying to output a value in the range [0, 1]?