I have a pandas data frame containing around 100000 observations of plant species and their age with additional numerical predictors (climate). I used tensorflow and keras to build a sequential model to predict species age. Here is the code for my basic regression network:

# Create model structure
model = Sequential([
    Dense(128,activation='relu',
          kernel_regularizer=regularizers.L1L2(l1=1e-3,l2=1e-4)),
    Dense(64,activation='relu',
          kernel_regularizer=regularizers.L1L2(l1=1e-3,l2=1e-4)),
    Dense(32,activation='relu',
          kernel_regularizer=regularizers.L1L2(l1=1e-3,l2=1e-4)),
    Dense(16,activation='relu',
          kernel_regularizer=regularizers.L1L2(l1=1e-3,l2=1e-4)),
    Dense(1,activation='relu')
])

# Compile the models
model.compile(optimizer=tf.keras.optimizers.Adam(),
                     loss=tf.keras.losses.mae,
                     metrics=['mae'])

# Train
random.seed(132)
model_trained = model.fit(X_train,
                          y_train,
                          epochs = 200,
                          validation_split = 0.15,
                          verbose = 0)

I one hot encode the species data and normalize all variables before training. There is around 110 species, so after one hot encoding them, I have a training dataset with around 900000 rows and 180 columns.

For training, I thought that including data for all the species in one data frame, will allow better performance, because the model is learning from more data at once.

The model does not show signs of overfitting and the validation, testing and training MAE is all very similar. Seems good!

However, when I try and make predictions, using different numerical data (future climate), the age predictions are all the same for different species. E.g. When I set up my new data frame for prediction, I encode the data frame to be for a specific species i.e. Species_1 is present, therefore all other species are not.

# Species_1 prediction
[[23.043112]
 [23.11334 ]
 [24.231022]
 [23.026756]
 [25.771097]]

# Species_2 prediction
[[23.043112]
 [23.11334 ]
 [24.231022]
 [23.026756]
 [25.771097]]

The fact that the predictions for different species are identical, makes me think that the network did not use the one hot encoded species information during training, or rather that the species information was not considered important in predicting age and therefore received a lower importance. I am by no means an expert in neural networks, and I am still learning.

My goal is to make age predictions for different species under different climate scenarios. How do I force the network to always use the species data during training? Am I wrong to use all species data in one data frame and should rather split the data frames per species separately? Should I rather use other machine learning algorithms (Random Forest)? Any tips would be appreciated.