I am trying to make a neural network on a dataset with 257 features and 1 target variable. My code looks like the following:
df = pd.read_csv('Training Data.csv', low_memory=False, index_col=0)
df = df.dropna()
dataset = df.values
X = dataset[:, 1:]
y = dataset[:, 0:1]
scalerx = MinMaxScaler().fit(X)
scalery = StandardScaler().fit(y)
X = scalerx.transform(X)
y = scalery.transform(y)
k = 257
X = SelectKBest(f_regression, k=k).fit_transform(X, y)
# imputer = IterativeImputer(verbose=2)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)
# X_train = imputer.fit_transform(X_train)
# X_test = imputer.transform(X_test)
model = Sequential()
model.add(Dense(k, input_dim=k, kernel_initializer='normal', activation='relu'))
model.add(Dense(k, kernel_initializer='normal', activation='relu'))
model.add(Dense(k, kernel_initializer='normal', activation='relu'))
model.add(Dense(k, kernel_initializer='normal', activation='relu'))
model.add(Dense(k, kernel_initializer='normal', activation='relu'))
model.add(Dense(k, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal', activation='relu'))
sgd = tf.keras.optimizers.SGD(learning_rate=0.0001)
model.compile(loss='hinge', optimizer=sgd)
model.fit(X_train, y_train, epochs=100, verbose=1, batch_size=16)
# model = GradientBoostingRegressor(random_state=0)
# model.fit(X_train, y_train)
res = model.predict(X_test)
res = scalery.inverse_transform(res)
MAE = []
MAPE = []
print('Neural Net Results...')
score = mean_absolute_error(y_test, res)
MAE.append(score)
print('MAE: ' + str(score))
score = r2_score(y_test, res)
print('r2: ' + str(score))
prediction = pd.DataFrame(res, columns=['predictions'])
prediction['actuals'] = scalery.inverse_transform(y_test)
prediction['percent'] = abs(prediction['predictions'] / prediction['actuals'] - 1)
print('MAPE: ' + str(np.mean(prediction['percent'])))
MAPE.append(str(np.mean(prediction['percent'])))
print(MAE)
print(MAPE)
prediction.to_csv('Standardized Prediction.csv')
I've tried different loss function and different learning rates and I continue to get an output that looks like this:
Epoch 1/100
348/348 [==============================] - 1s 677us/step - loss: 0.7974
Epoch 2/100
348/348 [==============================] - 0s 744us/step - loss: 0.7974
Epoch 3/100
348/348 [==============================] - 0s 712us/step - loss: 0.7974
Epoch 4/100
348/348 [==============================] - 0s 823us/step - loss: 0.7974
Epoch 5/100
348/348 [==============================] - 0s 815us/step - loss: 0.7974
Epoch 6/100
348/348 [==============================] - 0s 712us/step - loss: 0.7974
...
Neural Net Results...
MAE: 21.426221199575174
r2: -446.9135260782362
MAPE: 116766355636412.97
[21.426221199575174]
['116766355636412.97']
My output csv looks like this:
...
What might be the issue on why it isn't learning?